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ABSTRACT 

The research activities of project Intrex are 
directed toward the extension of man's intellectual reach by a new 
kind of control ever access to information. Ways are sought to 
improve the efficiency of catalog search by utilization of 
interactive computing techniques, and to provide rapid access to 
full-text displays by utilizing microfilm storage and facsimile 
transmission. User experiments, the central subject of the report, 
show a great deal of work needs to be done, but are mainly 
encouraging. The principal remedy for existing budgetary constrainsts 
which have called for drastic retrenchment will be found in the 
organizational domain. If the purposes of special interest groups can 
be met at an acceptable cost, utilization of these techniques can 
spread through interlibrary cooperation and networking to gradually 
form a multidisciplinary library network. (AB) 
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ERIC 



PROJECT INTREX 



Activity Report 



I. INTRODUCTION 

The research activities of Project Intrex are going forward at a time of deep- 
ening crisis in the major research libraries of this country. While struggling with the 
continuing problems that stem from the accelerating rate at which knowledge is published 
£ id used, librarians are now faced with budgetary constraints that call for drastic re- 
trenchment in many services. In these circumstances, the potential role of advanced 
technology is being discussed with a new sense of urgency. 

Where the primary challenge is economical and where near-term solutions 
are sought within existing concepts of library service, the promise of the new technology 
is not spectacular. We can expect to see a steady increase in the use of microfilm as 
an alternative to costly materials in their original form. Office copying techniques will 
be used for on- demand duplication of a growing volume of journal articles and other 
short items that will no longer be circulated and will thus remain accessible to other 
readers. Computers will be used wherever they can reduce the costs of acquisition, 
serials control, circulation, and other operations. 

Important and desirable as these techniques are, they will not, in themselves, 
restore our libraries to economic health. The principal remedy for today's afflictions 
will be found in the organizational domain. Interlibrary cooperation and "networking" 
will have to be raised to new levels of effectiveness by strengthening regional and 
national backstopping services. The goal of self-sufficiency will have to be abandoned 
by even the greatest of our libraries. A sensible fee structure will have to be developed 
to encourage the sharing of resources on a very much larger scale than today’s quasi- 
philanthropic loan arrangements. These organizational advances will be supported not 
only by the microfilming, duplication, and computing facilities of the individual li- 
braries, but also by irtte rlibrary technology that is now available, such as telephone and 
teletype communications, facsimile transmission, and data processing services that 
utilize centralized bibliographic resources. 

In contrast with these near-term contributions of new technology toward eco- 
nomic and organizational ends, the technological thrust of Project Intrex is directed 
toward a more distant and more ambitious goal. The objective of Project Intrex is to 
extend man's intellectual reach by giving him a new kind of control over his access to 
information. In pursuit of this objective, we are seeking to improve the efficiency of 
catalog searches by utilizing interactive computing techniques, and to provide rapid 
access to full-text displays by utilizing microfilm storage and facsimile transmission. 

We shall not be satisfied with moderate improvement in these services. We shall have 
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met the challenge before us only when library users who employ Intrex techniques have 
come to feel that their command over information resources is so powerful as to con- 
stitute a wholly new way of doing intellectual work. 

The user experiments which are the central subject of the present report are 
characteristic of the early stages of user involvement in a technical project, and a good 
deal of painstaking work remains to be done. But even at this stage the user experiments 
give strong encouragement to our belief that the aspirations of Project Intrex can and will 
be realized. 

We expect that the techniques of Project Intrex will find their initial use, prob- 
ably in narrow fields of science and technology, among special-interest groups that have 
a clear perception of the value of comprehensive and rapid access ro informal ion re- 
sources. If the purposes of these initial user groups can be met at a cost acceptable to 
them, we may see the utilization of these techniques spread to other groups and grad- 
ually form a multidisciplinary national network. The role of libraries as potential nodes 
in such a network is not yet clear; its definition will be one of the major tasks confronting 
the new National Commission on Libraries. 
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II. RESEARCH AND DEVELOPMENT ACTIVITIES (Electronic Systems Laboratory) 



A. STATUS OF THE PROGRAM 



Professor J. F. Reintjes 

Experimentation with the Intrex retrieval system under controlled conditions 
is consuming a major part of our time. During the Fall 197 0 academic term, we under- 
took a major literature-search operation with eleven students in a graduate subject 
offered by Professor Frederick McGarry. The motivation for the experiment was a 
term paper which each student was asked to write as part of his course requirements. 

The experiment, which extended over most of the fall term, has yielded an abundance of 
valuable information on several facets cf the Intrex retrieval system in cojupaji’itioii with 
more conventional sources of literature. 

Further analysis has been made of our Category-II Experiments on Indicativity 
which were reported in our two preceding Activity Reports. These experiments seek to 
determine the effectiveness of different types of catalog information as indicators of the 
usefulness of documents to users. Deeper insights into the data gathered during the 
experiments have helped us pinpoint with greater accuracy the significance of results and 
to guide us to possible loopholes in procedures which may be causing variations in results. 
Accordingly, a new series of Category-II expea orients has been initiated. The goal of 
this class of experiments is to determine the relative value of several types of document 
information — title, abstract, subject-matching phrases, and all index phrases — as 
indicators of usefulness of a document. Results of tn^se experiments are needed in order 
to be able to specify the quantity and kinds of information that should go into future large- 
scale machine -sto re d catalogs. Procedures for our new series have been tightened in 
order to eliminate possible influence of minor variations in experimental procedures on 
over-all results. 

A cost analysis of machine -oriented literature-retrieval systems has been 
initiated. The goal of this in-depth study is not only to evaluate operational costs of the 
Intrex system as they are now being incurred, but to project future costs of large-scale 
systems in light of specialized computer configurations, data-base, sharing over com- 
munications channels, anticipated technological advancements, and so forth, 

A major analysis of user reactions to machine-searching for literature through 
use of computer terminals of the types being employed by Intrex has been made. Our 
experiences were presented at a workshop sponsored by the American Fe da ration of 
Information Processing Societies at Palo Alto, January 14 and 15, 1971. A report on 
our study is in preparation. 

Opportunities to use the Intrex retrieval system on an open, noncontrolled, 
basis are being expanded. In addition to terminals in the Electronic Systems Laboratory 
and a teletypewriter and a combined catalog and full-text cathode -ray-tube console in the 
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M.I.T. Barker Engineering Library, we have a teletypewriter terminal in operation at 
the McKay Laboratory, Harvard University. At present only the catalog of literature 
stored in the M.I.T. -based computer can be searched at the Harvard location. Howeve 
the possibility of providing full-text access over the broadband microwave, coaxial- 
cable system which now links both campuses is being examined. The installation of an 
Int rex -terminal facility at the M.I.T. Center for Materials Science and Engineering is 
underway. The necessary communications channels should be in place within a month 
or so. 

In support of the preceding activities, development of the data base goes for- 
ward as does the maintenance and refinement of the various items of electronic equip- 
ments being employed in the Intrex system. Approximately 15, 000 items are now in- 
cluded in the computer -stored catalog. The second Intrex display terminal which we 
undertook to build six months ago is in the final debugging stage, and the remote text- 
access equipment has undergone several refinements to improve its reliability. 
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B. 



SYSTEM USAGE: EXPERIMENTS AND ANALYSIS 



Staff M imbers 



Mr. A. R. Rene rife Id 
Mrs. S. Brown 
Professor L. S. Bryant 
Miss M. A. Jackson 
Mr. P. Kugel 
Mr. S. G. Kuyamjian 



Miss L. T. Lee 
Mr. R. S. Marcus 
Miss V. A. Miethe 
Professor J. F. Reintjes 
Mr. J. O. Silvey 
Mrs. F. Spigai 



SUMMARY 



Major efforts were made during this reporting period in retrieval-system 
usage experiments and in analysis of the usage made of the Intrex facilities at the M.I.T. 
Barker Engineering Library and the Harvard University McKay Laboratory. Activities of 
the group also included: continuation of the Intrex Librarian-Training Program; an index- 
ing and retrieval comparison between Intrex and an abstracting journal; the design and 
execution of a new experiment measuring retrieval effectiveness involving a whole class 
of students; the further analysis and design of catalog indicativity experiments; an anal- 
ysis and evaluation of the Intrex user-system interface; and the installation and use of a 
new microfilm viewer. A detailed description of the work performed in these areas 
follows . 



INTREX FACILITIES IN OPEN ENVIRONMENTS 

The Harvard Facility . In early February, 1971, an IBM 2741 typev/riter 
terminal was installed at the Gordon McKay Laboratory of Applied Science of the 
Harvard University Division of Engineering and Applied Physics (DEAP). The Intrex 
catalog can now be interrogated at this terminal. The facility is available to all DEAP 
or other Harvard personnel and is an essentially unmanned operation except that the staff 
member in whose laboratory room the terminal is located acts as custodian of the instruc- 
tional material. In particular, the terminal is physically separated from the DEAP li- 
brary which is located in a separate building. 

The initial introduction of the facility to the Harvard community involved a 
lecture and demonstration to about twenty potentially interested staff and student per- 
sonnel. The first week of operation has been most encouraging. Users have, in gen- 
eral, mastered the system rather well, and they appear to be getting good, useful results. 
We suggest that these initial positive results tend to confirm our feelings of the importance 
of bringing terminals in close proximity to the working area of users with a real need for 
information in the data base. It also seems that the personal introduction via group dem- 
onstration and lecture was a successful tutorial device. 
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The Barker Engineering Library Facility . During the past six months the 
Intrex facility in the Engineering Library has been improved in a number of respects. 
First, the regularly scheduled hours were extended from three to six hours a day. Be- 
cause manning the consoles continuously for this extended length of time by Intrex advisers 
was not feasible initially, it was decided to try having the advisers available on a call 
basis only. This mode of operation gave us an opportunity to test how easy or difficult it 
would be for users to learn without any personal instruction. We did, in fact, observe 
that many users had a difficult time learning how to use the system to a high degree of 
proficiency without personal help. This fact, along with many others derived in whole 
or in part from the library experience, is outlined below in a later section on The User 
Interface . 



Subsequently, it was decided that the absence of continuous advisory service 
at the consoles inhibited the use of the facility to its full potential and also led to a loss 
oC information about the manner of system use that could be provided by the advisers. 
Consequently, it was decided to reduce the regularly scheduled hours somewhat and pro* 
vide full advisory service during that time. Starting in the spring term, the facility will 
be in operation daily from 11 AM to 4 PM and at other times by appointment. 

Several changes in the physical facilities were also made. The main display 
console was moved from the main reading room to the ring area of the library where 
the typewriter console is located. This change permits easy access to the hard-copy 
typewriter terminal by display-console users and also permits demonstrations to be held 
without disturbing other library user s . A draw K ack of the new location is that the console 
is in a somewhat isolated area and is not as often encountered by the casual passerby. 

A second change was the installation of the combined ARDS/TEXT display console whose 
physical description and high user acceptance are described below. A third change was 
the substitution of a Date! Model 30 portable typewriter terminal in place of the IBM 2741 
terminal This terminal is available on loan to users at their regular office or laboratory 
locations and so provides an additional kind of remote-access service. 

As a result of our experience with users, we have extended and modified our 
instructional aids. These extensions include new guides for the beginning user and new 
summary sheets for handy reference for the novice user. Attention has been given to the 
placing of placards to help the user with common problems, such as starting, and correct- 
ing errors, and to help locate various features of the system, including the other instruc- 
tional aids. A telephone has been provided to enable easier communication should addi- 
tional assistance be required. 

The average daily use of the system runs between five and ten persons daily. 
Although many of these users are students exploring the system, a fair number have a 
serious interest in the material contained in the system; we are particularly pleased with 
the number of repeat users. We have found it rather difficult to elicit comments from 
users while they are using the system but quite easy to obtain comments after they have 
used it. 




- 6 - 



9 



User computer dialogs are automatically monitored; complete records of all 
sessions are stored on disk and later printed out for analysis. A weekly summary of 
these monitor files is produced which contains information about individual sessions, 
system performance, schedule maintenance, system problems and their resolution, and 
user comments. Periodic meetings have been instituted at which both library and ESL 
personnel discuss experience with users, changes in the system, and suggestions for 
improvements in operation and experiments. 

INTREX LIBRARIAN -TRAINING PROGRAM 

The educational program to train Engineering-Library staff and ESL staff 
indexers as Advisers to operate the Intrex retrieval system has been revised as follows: 

An improved unit-content structure . Each unit contains logically 
related topics. Disparate jumps among unrelated topics have been 
removed. The general content outline emphasizes individual system 
features in relation to their role in the total system. There is also 
increased attention to progression from the more basic features to 
the more sophisticated system features. 

Measures of progress by instructional units . Test demonstrations 
by the trainee have been incorporated into the program at the end of most 
instructional units. Frequent testing of trainee progress now allows the 
instructor to institute reviews of topics at the time they are needed before 
the trainee moves on to other topics. 

Associated reading material . Relevant system documentation 
bearing on each instructional unit has been distributed to trainees and 
instructors. These readings are drawn fror/i the Intrex system over- 
view and reference guides, and from several Intrex reports and 
memoranda. 

The revised training program has been implemented with some changes from 
the initial procedures of last spring. Previously trained advisers are now participating 
as console instructors, with each instructor working with only one trainee. It has been 
and continues to be our experience that personalised "one-on-one" instruction with con- 
siderable "hands-on" console operation for the trainee best conveys system features, 
functions, and operations. The console instructors guide the console sessions of the 
trainees and they develop specific problems to be solved that pertain to the day’s topics. 
However, some of the instructional units are lectures or seminars covering more 
"theoretical" topics. Those seminars, and the administration of the end-of-unit review 
tests are given by the training-program developers. Based upon our previous training 
experience, the units of the revised program were developed to be covered in 25 daily 
sessions, each session lasting not more than two hours. 
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At the time this report was written, the first section of the revised training 
program was still in progress. One reference librarian and one staff indexer are cur- 
rently being trained. This revised program is running far more satisfactorily to both 
instructors and students alike than did the initial program. 

An outline of the instructional units is given below. The end -of -unit tests 
have not been listed. The tags M C U and 11 L" refer to units with console operation or 
lectures, respectively. Some units require two or three two-hour sessions, whereas 
others can be covered in less than a session. 

Outline of Instructional Units for 
Librarian-Training Program 

KEY: L - Lecture Session C - Console Session CL - Console and Lecture 



Unit A. 


Overview of System Use 


L 


1. History and Experimental Objectives of the Intrex System 


C 


2. Basic Steps in Intrex Use: Search; Catalog Output; Text Output 


C 


3. External Mechanics of System Use: 

Log-in and out; Console Operations (ARDS, 2741); Typing Errors; 
Interrupt; The Commands HOLD, BEGIN, LOG, EXIT 


CL 


4. Text-Facility Us e 


Unit B. 


Subject Searches 


CL 


5. Subject Indexing 


C 


6. Subject Searches : Commands SUBJECT, RANGE, COUNT, 
OUTPUT match subject 


L 


7. Subject-Search Mechanisms I: Word Stems, Inverted File, Synonyms 


CL 


8. Subject-Search Mechanisms L: Lists and List Names 


CL 


9. Subject-Search Mechanisms III: Combining Lists and Other Techniques 
for Overcoming Synonym and Matching Failures; Bookkeeping Problems 
with Lists 


Unit C. 


Other Inverted-File and Basic Searches 


C 


10. Author Searches ; Title Searches ; Document Number Searches 


Unit D. 


Catalog Information and its Output 


L 


11. Catalog-Record Structure ; Tagging; Codes; Abbreviations 

12. Catalog-Record Fields and their Output 


Unit E. 


Search of Uninverted Fields 


C 


13. RESTRICT Command and "Eyeballing 11 


Unit F. 


Other System Features 


C 


14. Other Commands: INFO; LONG; SHORT; TIME; COMMENT 


L 


15. Instructional Features On-line: Info; System-Dialog Structure ; 
Error Messages 


0 

ERIC 
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Unit 



L 16. Instructional Features Off-line: Overview; Guides; Cards 
L 17. Data-Base Coverage and Selection 

G. Roles of the Adviser 

L 18. Strategies and Strategy Aids 

L 19. Roles of the Adviser: Instructing and Observing Users 

C 20. Final Test: Demonstration of the Entire Intrex System 



INDEXING COMPARISON: INTREX AND ABSTRACT JOURNAL 

Background . In the previous two Semiannual Reports several Category-I 
experiments were described in which Intrex indexing and retrieval were analyzed with 
respect to standard bibliographies. One of these experiments involved an experimental 
subject designated ES 1 who had prepared a bibliography on the properties of the com- 
pounds RbMnF^i KMnFgi CsMnF^i and TlMnF^, especially with regard to the property 
antiferromagnetism . It was reported that Intrex searching of this topic gave a recall 
figure of 0.87, precision of 0.90, and a discovery ratio of 0,22 -- that is, relevant docu- 
ments found by Intrex and not in the bibliography were 22 percent of all Intrex-found doc- 
uments. There was some speculation about the relatively low discovery ratio, especially 
in view of the supposedly much deeper Intrex indexing than that of Physics Abstracts, the 
abstracting journal, and the fact that about half of the discovery set was attributed to 
"clerical-error" omission by the bibliography authors, and not to the deeper indexing of 
Intrex. A subsequent detailed analysis, summarized below, of the indexing of Intrex and 
Physics Abstracts on this topic has shed considerable light on this and related questions. 

Analysis . Some 56 documents from the Intrex data base were selected for 
study. A total of 51 of these had been retrieved by Intrex in the searches conducted during 
the experiment. The searches were on the chemical-compound formulas. Five others 
had been in the bibliography of ES 1 and in the Intrex data base but not retrieved by the 
above search strategy. Five of the 51 retrieved had been judged not relevant by the 
author cf the bibliography. 

As the first part of the new analysis, these 56 documents were scanned to deter- 
mine if any of the four chemical formulas or a word with the stem antiferromagnet exist 
in the title of abstract of each of the documents. There were 128 occurrences of these 
five terms in the titles and abstracts. The breakdown by term and location is given in the 
following table. 
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TERM 



Location 


Rb MnF^ 


KMnF^ 


CsMnF^ 


TlMnF 3 


Antiferro- 

magnet 


Totals 


Title 


21 


9 


0 


4 


17 


51 


Abstract 
(also Title) 


18 


6 


0 


3 


16 


43 


Abstract 
(not Title) 


5 


11 


2 


0 


16 


34 


Totals 


44 


26 


2 


7 


49 


128 



The above figures refer to the existence of a term within the title and/or abstract of a 
document; multiple occurrences of a term within an abstract or a title of a single docu- 
ment are counted as only one occurrence. It may be observed that for 51 occurrences 
of terms in title, the same terms also appeared 43 times in the abstract. We may also 
note that the effect of adding abstract words to title words has added about 67 percent 
(34/51) to the number of different term- in-document occurrences. We ignore the 43 
co-occurrences of terms in abstracts since they already are covered by title words. 

Let us now look at these 85 (51 +34) instances of terms in documents to see how 
they were indexed. All 51 title terms were indexed in Intrex, of course, inasmuch as all 
non-common title words are regularly indexed by Intrex. Physics Abstracts (PA) indexed 
44 of these terms (86 percent). All of the 51 terms came from documents rated relevant 
to the topic by the author of the bibliography. There was a marked difference between 
PA indexing of the chemical compounds only, where 33 of the 34 (97 percent) of the title 
occurrences were indexed, and the indexing of the property antiferromagnetism , where 
only 11 of 17 (65 percent) of the abstract-only terms were indexed as such. A document 
was considered indexed under the term antiferromagnetism if it appeared under any head- 
ing with that word, ora word with the same stem, in it; namely, the PA headings Antiferro- 
magnetic resonance, Antiferromagnetism, and Magnetic properties of substances/anti- 
fe r romagnetic . 

With respect to the 34 abstract-only words, Intrex indexed 27 (80 percent) of 
them. PA indexed just half (17 of 34, or 50 percent) of these same occurrences. The 
greater emphasis on chemical compounds in Physics abstracts is also found here, but 
to a lesser degree; 56 percent (10/18) of the chemical compounds were indexed by PA, 
while only 44 percent (7/16) of the antiferromagnetism were indexed by PA. Three of 
these 34 occurrences came from two documents which were considered not relevant to 
the bibliography. Intrex indexed 2 of these 3, and PA one . It can be determined from the 
figures that the overall coverage of PA indexing of title and abstract words is (44+ 17)/ 

(51 + 34) = 61/85 = 72 percent. Intrex coverage is (51 + 27)/85 = 92 percent. 
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It is instructive to consider what range numbers are associated with the 
Intrcx index terms containing one or more of the five words under study. The table 
below gives the range number of abstract-only terms indexed by Intrex in each range, 
which of these were indexed by PA, and how many were frcm irrelevant documents. 

Where a word was indexed in more than one Intrex term, only the term with the higher 
range number was considered, where the order of importance from high to low is taken 
as 5 (title), 1, 2, 3, 4, 0. (Note, of course, there are no title words in abstract-only 
terms.) The seven abstract-only words that were not indexed by Intrex are also indicated. 





Intrex Range 


Number 


Not Indexed 
by Intrex 


1 


2 


3 


4 


0 


Intrex Indexing 


12 


7 


4 


4 


1 


7 


PA Indexing 


8 


4 


0 


1 


0 


1 


Irrelevant 


0 


1 


1 


0 


0 


1 



It can be seen from the results of the analysis that Intrex indexing to a depth of 
range 1 is roughly equivalent to PA indexing in that Intrex title and range 1 terms cover 
some 63 (51 title plus 12 range -1 terms) of the 85 terms, whereas PA indexing cover 61 
of the 85 terms. Notice, also, that the abstract-only terms indexed by PA tend to be in- 
dexed under the more important range numbers in Intrex, indicating that the range num- 
bers are properly serving their intended function in that the PA- indexed terms should be 
those with the more important relationship to the document. 

A sample of 19*W citations indexed by PA indicates that between 3 and 4 headings 
are assigned to each document; a total number of 15-20 words per document are involved in 
subject headings on the average. This, again, can be characterized as "expanded title" 
depth indexing and related to the Intrex title and range- 1 indexing. 

In addition to the title and abstract terms that were indexed, Intrex indexed the 
five topic words on 1 5 occasions when they appeared in neither title or abstract. On only 
a single occasion did PA do likewise. Five of the 15 Intrex terms came from documents 
that were judged not relevant to the topic, the one PA term was from a relevant document. 
The breakdown by range number follows: 



Intrex uses a range number after each subject-index phrase to indicate the extent of 
the document to which the phrase applies. For a detailed description of range number 
see: 15 September, 197 0 Intrex Semiannual Activity Report, p. 17. 



Intrex Range Number 





1 


2 


3 


4 


0 


Total Terms 


0 


3 


7 


5 


0 


Irrelevant 


0 


0 


4 


1 


0 



Note here, also, that the range numbers help differentiate more important, or relevant, 
documents in that all of the irrelevant documents had low range numbers. 

Let us sum the terms from the 51 relevant documents in three categories: 

(1 ) title terms, 51; (2) abstract-only terms, 34 - 3 = 31 ; text-only terms, 15-5=10; 
for a grand total of 92 terms from relevant documents (8 terms were from non-relevant 
documents). Taking this as a kind of ’’recall base for terms, ” we may then calculate 
overall recall and precision figures as follows: 





Intrex 


PA 


Recall 


(58 + 26 + 10)/92 = 0. 95 


(44 + 16 + 1 )/92 = 0.67 


Precision 


87/(87 + 2 + 5) = 0. 93 


61/(61 +1) = 0.98 



Breaking this down by Intrex range number, we have 



Recall 
Precision 

These figures are plotted on Fig. B-l. The estimated cumulative percent of index words 
for a given range is taken from data-base averages, as described in the 15 September 1970 
Semiannual Report. 

The number of terms indexed by Intrex but not by PA is 92-61 = 31; the number 
indexed by PA but not Intrex is just one . The discovery ratio for Intrex is then 31/87 = 0.36. 
The number of relevant documents not indexed by PA under any of the five terms was de- 
termined to be 10. Thus the discovery ratio on a document basis is 10/51 = 0.2 0. 

Interpretation . The Intrex discovery ratio of 0.20 with respect to Physics ab- 
stracts on a document basis is more in line with our previous results — compare this 
with the discovery ratio of 0.23 reported for the bibliography on "fatigue" with respect 
to the journal Metals Abstracts. One reason that the previously reported figures appeared 
so variant is that the Intrex discovery ratio of 0.11 on the antiferromagnetism bibliography 
was taken with respect to the bibliography itself, whereas the figures given above are with 



Intrex Range Number 



5(title) 


1 


2 


3 


4 


0 


51/92-= 0.55 
51/51 = 1.0 


63/92 = 0.69 
63/63 = 1.0 


72/92 = 0.7 8 
72/73 = 0.99 


78/92 = 0.85 
7 8/84 = 0.93 


86/92 = 0.94 
86/93 = 0.93 


87/92 = 0.95 
87/94 = 0.93 



Cumulative figures for all range numbers up to, and including, range given. 




- 12 - 

15 



respect to abstract-journal indexing only* Thus, the lower figure of 0.11 resulted 
partly from the fact that the author of the bibliography went beyond simple searches in 
one abstract journal. 




ESTIMATED CUMULATIVE PERCENT 
OF INDEX WORDS 



Fig. B-l Recall as a Function of Depth of Indexing. 

Another point to consider is the much higher discovery ratio on a term basis 
(0. 36) than on a document basis (0.2 0). This reflects the fact that any search on a strat- 
egy more sophisticated than just ?, single word — here a disjunction of five one-word 
terms — will tend to counteract isolated indexing deficiencies. This interpretation 
raises again the question of why the discovery ratio on the single term "fatigue" appears 
to be so low, and this question will be further investigated. 

The paradox as to why the discovery ratio on a term basis (0.36) appears so 
small in view of the supposedly much greater depth of Intrex indexing with respect to 
Physics Abstracts can now be resolved. The statement of the paradox is that Intrex 
has about 80 entry points (subject words) per document whereas the PA index has only 
about 4 ; therefore, the discovery ratio should be about 20. However, as noted above, the 
PA index entry point is usually a multi-word phrase and the actual number of words repre- 
sented may average about 15 to 2 0. Also, the Intrex indexing is purposely highly redun- 
dant on a word basis — in order to allow subject terms to "stand alone" for greater 



indicativity for the user and in order to provide a linking mechanism for greater search- 
ing precision by the Boolean command WITH. A recent analysis of the Intrex inverted 
files indicated that the redundancy on a word basis over the whole data base was on the 
order of 2. 5, so that of the 80 subject words in the catalog record for the average docu- 
ment there may be only about 3 0 different word-stem types. This implies a discovery 
ratio of about (30 - 20)/30 = 0.33, which is very close to the 0.36 figure we actually ob- 
served in this experiment. 

Summary . The analysis described above has helped to resolve certain ques- 
tions about the comparison of Intrex indexing with respect to other kinds of indexing, 
especially in regard to how depth of indexing affects retrieval effectiveness. This analysis 
will serve as a pattern for additional analysis which will attempt to further quantify the 
factors affecting retrieval effectiveness for a greater variety and number of searches. 

One particular aspect of future analyses is to explain why Intrex title and range -1 terms 
for this experiment seemed to have a significantly greater effectiveness when compared 
to lower range terms than reported for some previous experiments. 

THE CLASS EXPERIMENT ON RETRIEVAL EFFECTIVENESS 

Nature of the Experiment. An experiment on retrieval effectiveness was con- 
ducted during the fall semester with the full cooperation of Dr. Frederick McGarry, Pro- 
fessor of Civil Engineering, in conjunction with a term-paper assignment for eleven grad- 
uate students taking his subject "Strength of Structural Materials." The data from this 
experiment are now being analyzed and interpreted. We discuss here the objectives of 
the experiment, its implementation, and some initial results and observations. 

The basic purpose of this experiment is a comparative study for the same search 
topic of retrieval effectiveness using Intrex and traditional library sources. The effects 
on retrieval of indexing features of Intrex and of the several library sources, and the 
searching strategies employed in all of the sources are of particular interest. A secon- 
dary purpose is the evaluation of the interactive features of the on-line Intrex system. 

The quantitive measures employed are: recall as a function of effort, the latter being 
measured by time invested and number of searches; and recall and precision for indi- 
vidual searches and for combinations of searches. An important feature is that we are 
able to provide comparisons and averages of several searchers working independently 
on the same problem, and among several search groups, each assigned a different topic . 

In addition, the effects of preceding Intrex searching by library searching, and the re- 
verse, are being studied. 

Implementation . The implementation of the experiment followed these major 
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steps : 



1. The active cooperation and approval was obtained from 
the professor who is the leader of one of the five research 
groups whose interests comprise the thrust of the Intrex 
data base. 

2. A general introduction to the experiment was given to the 
eleven students in the class. The professor and the students 
together suggested fourteen potential topics for the class term 
papers. Our main interest was in the literature- searching 
methods and the materials gathered for the papers. 

3. A quick test on the breadth of the fourteen potential topics 
was made in Intrex and in major abstracting sources. 

Three topics were selected which were the least likely 

to bias results because of a simplistic search procedure, 
or by too few or too many references in Intrex and other 
sources. These topics are: 

(a) Irradiation embrittlement of metals. 

(b) Effect of micro structure on the fracture toughness of 
steels . 

(c) Effect of processing history on the fracture toughness 
of aluminum alloys. 

4. Each student selected one of the three topics. The topic group 
sizes were 4, 5, and 2 in the order of the topics listed above. 
Half of the students in each group began literature searching with 
Intrex, whereas the other half began literature searching with 
library sources. The second part of their searches in the other 
source did not begin until completion of the first part. 

5. Each student was given a set of written general instructions in 
mid-October 197 0. He also received copies of guides to the 
several campus libraries, including the Engineering Library. 

He was asked to contact Intrex personnel when he was ready to 
begin his Intrex searches. The completed term papers, refer- 
ence bibliographies, source listings, and a 500-word critique 
of their use of the Intrex system were due shortly before 
Christmas . 

6. The general instructions requested the students to format 
their term papers along the lines of a paper containing a 
comprehensive bibliography which might appear in an 
•'annual review" volume covering their topic. In particular. 
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the literature appearing since January 1967 was to be sur- 
veyed, although a brief introduction might also cite a few 
classic papers on the topic. 

7. When a student was ready to start his Intrex search, an 
Intrex analyst gave him a demonstration (about one -half 
hour in length) at a console using sample searches un- 
related to his search problem to indicate general system 
features. The student also received at that time hard copies 
of a system overview and a more detailed reference guide to 
the Intrex system. A ten-minute interview covering the 
student's general educational background and previous computer 
and library expe rience, preceded the demonstration. If the 
student had previously completed the library part of the assign- 
ment, then the general interview was followed by a debriefing of 
those search events and results. At the conclusion of the 
demonstration itself, the student was asked to arrange a spe- 
cific time to begin his first Intrex search session. 

8. A student was allowed as many Intrex search sessions as he 
felt he needed. An Intrex analyst was present during each 
session to assist the student with technical problems and 

to record observational data. The analysts were instructed 
to encourage the students to use the Intrex system and re- 
lated instructional materials as much on their own as possible. 
This is similar to the mode of operation of Category-Ill 
experiments described in previous reports. 

9. A short debriefing about results to date and conscious changes 
in the approach to the next Intrex search session preceded 
each subsequent session. A tape recording of the complete 
dialog between student and analyst was made for each session 
which also included the analyst's interpretive comments about 
uve of the text-access system and system malfunctions. Com- 
plete monitor files of all catalog system transactions were 
maintained. No charge was made for requests for microfiche 
and hard copies of text of documents found through the Intrex 
system. 

10. For the conventional-library search work, each student was 
requested to keep data on the time he expended in searching 
and examining literature sources; he was also asked to docu- 
ment the headings searched under in those sources and to list 
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potentially useful references from the headings. A worksheet 
to record the data was provided. If the library search was 
done after completion of the Intrex search, a debriefing on the 
library-search effort was separately arranged. 

11. Each Intrex search session plus the library search was docu- 
mented in memorandum form by an analyst from data pro- 
vided by the monitor files, recordings, worksheets, and the 
analyst's own recollections of observations and impressions. 

12. Copies of the final term papers, the Professor's comments 
on them, and the 500-word critiques of the Intrex system 
were secured. 

13. After concluding the formal experiment, we invited the students 
to attend an informal meeting at which time we discussed the 
larger goals of the experiment and presented some of our initial 
findings to them. V/e encouraged and received many additional 
comments about the experimental procedure and about the Intrex 
online retrieval system. 

Initial Results and Observations . The students, in general, learned to use the 
system readily and effectively. The controlled monitoring and observation of their be- 
havior was an important factor in developing the analysis of the user- system interface 
described in the section below. A gross picture of the primary statistics describing the 
experiment is given by the Table B-l. These statistics represent just the very initial 
results on this experiment. Further analysis and interpretation of the results are in 
process and will continue. 

It is tempting to use these statistics to get estimates of the effectiveness of the 
Intrex and library searches as measured by the number of references found per hour. 
However, several considerations make it difficult to do this on a simple basis. In the 
first place, the time spent with Intrex included text reading on the display CRT screen 
whereas time spent in the library, in general, did not include reading text. Secondly, 
the references not obtained from Intrex were not always found during library search 
time. For example, references included the class textbook and relevant technical papers 
landed out in class, as well as lists of reports supplied on letter request by government 
agencies. A more thorough analysis will have to be done before sound figures can be 
derived. 

Roughly speaking, however, it appears that when the above considerations are 
taken into account, the effectiveness factor would favor Intrex searching somewhat over 
standard library searching. The main reason, apparently, that the much greater speed 
and ease of Intrex searching did not weigh more heavily is that the general literature for 
these topics is much more voluminous than the coverage given the topics in the Intrex 
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Table B-l 



Statistics on the Class Experiment 













STUDENT 














Statistic 


A1 


A2 


A3 


A4 


B1 


B2 


B3 


B4 


B5 


Cl 


C2 


Total 


Average 


No. of Intrex Sessions 


3 


1 


2 


2 


2 


1 


1 


2 


2 


1 


5 


22 


2 


Time of Intrex Sessions 
(hours at the console) 


5.2 


2.5 


4.6 


2.5 


4.8 


7.2 


2.5 


3.2 


6.5 


1.7 


13 


53.7 


4.9 


First Searched Intrex (I) 
or Library (L) 


I 


L 


I 


L 


I 


L 


L 


I 


I 


I 


L 


61, 5L 




Time Spent in Library 
(hours) 


3 


1 + 


15 


8.5+ 


4 


8+ 


35 


3 


1 + 


1 + 


27 


106+ 


9.6 


Length of Term Paper 
(pages) 


1 3 


18 


24 


24 


17 


16 


41 


27 


25 


9 


30 


244 


22.5 


No. References in 
Term Paper 


12 


12 


39 


51 


19 


33 


37 


20 


31 


4 


16 


274 


25.0 


No. References in 
Term Paper Found 
through Intrex 


4 


2 


4 


9 


7 


13 


0 


6 


18 


2 




70 


6.4 



NOTES 



(1) + indicates an additional amount of time was involved but this hasn’t been fully- 
determined yet. 

(2) .Student-code prefixes A, B, and C refer to the three topics of Section 3 above. 

data base. The students were essentially unanimous in their comments, as expressed in 
their critiques, on the inherent great potential of an Intrex-like system and the corollary 
point that all that was needed to achieve this potential was a more comprehensive data 
base. 

It is interesting to note that several of the students have returned to the Intrex 
system, and introduced fellow students to it, since the end of the experiment. They have 
reported useful results in problem areas separate from the class topics in so doing. 

Our preliminary analyses tend to show that students who searched in Intrex 
before the library found a somewhat higher percentage of their references through Intrex. 
This is probably due to the factor of the students in wishing to finish up their term papers 
as quickly as possible. For example, the fact that student B3 shows the complementary 
effect in that his use of none of the references he found in Intrex apparently reflects the 
fact that he had finished his term paper before using Intrex. 

A statistic suggesting that Intrex has good recall capabilities is that of 7 3 
references used which existed in the Intrex data base, 7 0 were actually found through 
Intrex searching. 

There is evidence that the speed and ease associated with Intrex use by the 
students resulted in large measure from the unitary appearance of the data base to the 
user. That is, a single search request in Intrex will result in a list of documents which 
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match over the entire time span of the data base; also in-depth catalog or text output on 
these documents is immediately available at the user's station. In contrast, searching 
in abstracting journals involves looking through many different issues under several 
headings — phrase decomposition and co-ordinate word matching is lacking. Further- 
more, the immediate result is usually only a document title; procurement of the corre- 
sponding abstract and text requires two additional locating operations per document. 

Preliminary results suggest that the greater depth of Intrex indexing did not 
result in as much additional recall as we would have predicted. This appears to result 
largely from the fact that the students — for lack of time or other reasons, — chose 
not to be nearly so comprehensive and exhaustive in their searching as we hoped they 
would be. This reinforces the notion — important to measures of overall system utility 
— that many users prefer a few "good" documents to a comprehensive list. 

From other initial investigations it appears that there is often little difference 
in search results between the free-vocabulary aspect of the Intrex indexing and the 
controlled-vocabulary aspect of the abstract-journal type indexing. This may be due to: 
(1) the abstract journal primarily using title and some abstract words as the basis for 
indexing — essentially similar to Intrex, and (2) the key words directly associated with 
the student's topics were all fairly standard metallurgical terms, not too specific, and 
hence, part of the controlled vocabulary. If this result is found through further analysis 
and experiment to be general, it may be that the superiority of free -vocabulary indexing 
would rest on grounds such as ease of manual indexing, adaptability to automatic indexing 
and avoidance of controlled-vocabulary problems such as teaching searchers the vocabu- 
lary or updating the vocabulary by system analysts. 

THE CATALOG INDICATIVITY EXPERIMENTS 

Summary . The catalog indicativity experiments, previously designated as 
the Category-II experiments, are designed to test the effectiveness of different types of 
catalog information as indicators of the usefulness of documents to users. Since com- 
pletion of our first experimental program (Series A), work on these experiments has 
progressed on two major fronts. First, more data have been obtained from experimental 
subjects (ESs) 2G and 25, who participated in Series A, and an in-depth study of their 
responses as ESs in Series A lias been made. Second, an improved design for the catalog 
indicativity experiments has been devised and is being utilized in the Series B experi- 
mental program, which is now underway. 

Further Experimentation on Previous Experimental Subjects . ESs 20 and 2 5, 
both of whom are Ph. D. candidates in the Department of Engineering and Applied Physics 
at Harvard University, were chosen for additional study because their responses in Series 
A were anomalous in certain respects. For £S 20, titlec appeared more indicative of the 
usefulness of documents than were any of the other four types of catalog information 
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presented — abstracts, subject phrases, author, and the combination of title, author, 
location, and matching subject phrases. ES 25 was unusual in that he gave value ratings 
of *4' { "cannot make a judgment") to approximately one quarter of the subject-phrase 
occurrences . 

In this follow-up stud/ ESs 20 and 25 were asked to reexamine the fields and 
full texts of documents whose ratings of one or more fields differed from the full-text 
rating. They we r e asked to explain, if they could, why each discrepancy between field 
and full text ratings occurred — whether some information about the document was 
missing from the field, whether the field was misleading, whether the field rating was 
actually borderline, or whatever. From the explanations of the ESs, we were able to 
perform an informational-factor analysis similar to the one reported in the 15 September 
197 0, Activity Report. The recent analysis, however, differs from the previous one in 
two notable respects: information factors were assigned to individual "discrepant" fields, 
rather than to documents as a whole, and borderline cases were not always given an infor- 
mation factor of *0', but were given 1 1 / 2 1 when there was evidence that the catalog infor- 
mation was responsible for the borderline rating. Following is the code used to assign 
information factors to fields: 

0 A variational factor relating to the judgment of the ES 
was responsible for the discrepancy. 

1/2 The field rating differed from the full-text rating by 

1/2 point (borderline case), and the catalog information 
was in some way responsible for the discrepancy (infor- 
mation missing or misleading); or the ratings differed 
by one point, but a combination of informational and 
variational factors was responsible for the discrepancy. 

1 The field rating differed from the full-text rating by one 
point, and the catalog information was in some way re- 
sponsible for the discrepancy. 

Table B-2 below represents the total information factors per catalog field 
divided by the total number of discrepant ratings per field. The author field is omitted 
because all the discrepant ratings in that field were attributable to the field's low infor- 
mation content. Notice that the catalog information is responsible for approximately 
60 percent of the discrepancies between field and full-text ratings; the other 40 percent 
are attributable to variational factors of the ESs. 

Table B-3 displays data on indicat ivity (fraction of cases in which field ratings 
concur with full-text ratings) and "adjusted" indicativity. The latter measure is obtained 
by adding to the concurrent ratings those discrepant ratings that are attributable to varia- 
tional factors of the ESs. The combined indicativity levels for both ESs were raised over 
a range from 3 percent to 10 percent, with the most dramatic improvements for abstracts 
and subject phrases. 
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Through similar analyses yielding adjusted indicativity scores, we be- 
lieve we can gain a more accurate picture of the actual value of the catalog fields as 
indicators of document usefulness than we could with raw indicativity scores alone. 



Table B-2 

Fraction of Discrepant Ratings between Field and Full Text 
which can be Attributed to Inadequacies of Catalog Information 



Expe rimental 
Subjects 


Title 


Title, Author, 
Location, Matching 
Subject Phrases 


Abstract 


Subject Phrases 


All Fields 


ES 20 




£(•78) 


-£<.71) 


£(.75) 


£(- 73 > 


ES 25 


o 

o 

0'|CT' 


^(.67.) 


-£-(•2°) 


■±^(.375) 




Both ESs 


-£(.87) 


££(- 72 > 









Table B-3 

Raw and Adjusted Indicativity Scores 



Expe rimental 
Subjects 


Title 


Title, Author, 
Location, Matching 
Subject Phrases 


Abstract 


Subject Phrases 


All Fields 


ES 20 

Indicativity 


£<• 70) 


£(.50) 


£ ( - 61) 


£(.60) 


£(.61) 


ES 20 Adjusted 
Indicativity 


£<- 80 > 




£(.72) 


£(-™> 


£.71) 


ES 25 

Indicativity 


-£<•74, 


£(•77) 


£(.66) 


£.63) 


103, 711 


ES 25 Adjusted 
Indicativity 


£-(•74) 




£(•93) 


30 1/8, oi , 
35 (- 86 < 


122. 8 , Q .. 
~nsr ( - 84) 


Both ESs 
Indicativity 


£-(.73) 


£<- 69 > 


£.64) 


£.62) 


149 / 4 7 v 

■zzr (,67) 


Both ESs 
Adjusted 
Indicativity 


£(• 76 ) 


-££-(.78) 


£•89) 


3^.80) 


176.8, Qm 
' 2’22" ( - 80) 



Utility of Fields as a Function of Number of Words . As stated in previous 
Reports, we have formulated a length hypothesis for catalog data, which states that as 
field length increases, indicativity increases, but indicativity per word decreases; this 
is a statement of the law of diminishing returns as it applies to catalog data. We tested 
this hypothesis with respect to ESs 20 and 25, who, as indicated above, provided us with 
certain anomalies in their results to our Catagory-II experiments. 
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A count was made of all words (not solely content words) in the four fields 
under test of documents evaluated by ESs 20 and 25. In Fig. B-2 average field length 
is plotted against indicativity. The figures show that, after adjustment of the data, the 
anomalous results on these two ESs with respect to the length hypothesis tend to deviate 
less from the length hypothesis than do the raw data. It may be remarked that a similar 
effect was observed with analysis of ES 12 and ES 23 reported earlier. There are, of 
course, deviations from a completely consistent adherence to the hypothesis as would be 
expected in any probabilistic situation. We must consider further the statistical evidence, 
especially in the new, better-controlled, Series- B experiments, in order to determine 
the extent of adherence to the length hypothesis and the nature and reasons for deviations 
from it , 




Fig. B-2 Indicativity as a Function of Average Field Length. 



Independent investigations generally supporting the length hypothesis have been 
reported in: 

(1) Hagerty, Katherine, ’'Abstracts as a Basis for Relevance Judgment", Working 
Paper No. 380-5, University of Chicago Graduate Library School, February, 1967. 
(PB 174 394). 

(2) Saracevic, Tefko, "Comparative Effects of Titles, Abstracts and Full Texts 
on Relevance Judgments", Proceedings of the 1969 ASIS Annual Conference, 

October, 1969. pp. 293-299. 
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There are, of course, other factors beyond just length that affect the util- 
ity of catalog fields as indicators of document value. Some of these were highlighted by 
the interviews with ES 20 and ES 25. For instance, titles performed quite well for ES 20, 
even though they had the smallest number of words. ES 20 remarked that he would have 
preferred to go immediately from titles to full text, primarily because he could never be 
certain that a document would be useful until he examined the document itself. ES 25 
also expressed preference for certain kinds of catalog information. He found abstracts 
most useful because of their summary nature; subject phrases, on the other hand, were, 
in his opinion, too long and disconnected. Their pastiche quality made him unsure whether 
documents would be useful, and hence he was unable to make judgments on a substantial 
portion of subject- phrase occurrences. It should be noted, however, that over the course 
of the catalog indicativity experiments to date, the subject-phrase field has appeared to 
be at least as useful as the abstract field. Nevertheless, we shall further consider in 
our analysis whether the subject-phrases field may cause users more difficulty — say 
in terms of increased time to make a decision. 

Improved Experimental Design . A revised experimental design has been 
drawn up and is currently in use for our indicativity experiments. Substantial changes 
from the Series-A design are in the areas of: selection of experimental subjects, state- 
ments of search topics, selection of documents, and number and type of experimental 
sessions. Some less consequential modifications of Series A procedures have also been 
made. 

Selection of Experimental Subjects . We will continue our 
analysis of doctoral students who wish to obtain information 
relevant to their thesis topics. We will, however, also include 
other types of users in our study — undergraduates, graduates, 
faculty, and s»* aff members. Th<ise various types will compose 
a heterogeneous group, whose results will be compared to those 
of the thesis students. Our greater diversity in sampling will 
permit us to draw conclusions about the indicativity of catalog 
fields for various user populations. 

Statements of Search Topics. ESs will be encouraged to 
write short statements about the nature of their research topics 
and about the kind of information they wish to extract from the 
Intrex data base. If their research topics are clearly formulated 
before the experiment, the basis on which they make evaluations 
will hopefully remain more stable during the course of the exper- 
iment. Variability will be reduced, and hence the validity of 
our measures will be improved. 
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ESs will also be advised to prepare a list of search words 
which will be used to retrieve articles from the data base. 

Selection of Documents . Each ES will be given approximately 
20 documents to evaluate. We have found that evaluating more 
than 20 documents can be quite time-consuming; with large 
numbers of documents, ESs tend to make their evaluations 
more speedily, and hence less carefully. 

The 20 documents will be selected from all documents 
retrieved by use of a standardized technique. Specified 
numbers of documents will be picked randomly from docu- 
ments retrieved by certain search words, until the quota 
of 20 documents has been filled. With this technique we hope 
to obtain a wide spectrum of documents, ranging from highly 
useful to completely unsuitable. 

Number and Type of Experimental Sessions . ESs will 
participate in two experimental sessions — one in which they 
evaluate field information and another in which they evaluate 
full texts. The sessions will take place in the Intrex Laboratory 
under controlled conditions. After these sessions have been 
completed, ESs will be interviewed for background information 
and for their reactions to the experiment. 

Following the interview, a procedure similar to the one described 
above, in the section on ESs 20 and 25, will be followed. ESs will 
be given the fields and full texts of ten or fewer documents having 
discrepant ratings between field and full text. They will be asked 
to explain, if they can, why the discrepancies occurred. At the 
same time ESs will be given a list of the fields in the catalog 
record, with a short description of the general content of each 
field. They will be asked to check fields whose type of infor- 
mation would have been most helpful to them in judging the use- 
fulness of their documents. They will also be presented with the 
actual catalog records of three documents which were used in 
the experiment) they will be instructed to check the fields which 
contain the information that -would have been most helpful to them 
in evaluating the utility of those particular documents. The ESs 1 
responses about the catalog record will help us to determine which 
fields, besides those already isolated for study, are most useful 
as indicators of document utility, and which additional fields, if 
any, should be included in the catalog indicativity experiments for 
evaluation by ESs. 
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Other Modifications of Series -A Procedures . The numerical- rating 
scheme has been changed as follows to measure the degrees of a single variable, 
namely, usefulness: 

1. The article is highly **seful. 

2. The article is somewhat useful. 

3. The article is not useful. 

The '4' rating — "cannot make a judgment" — has been omitted. However, ESs will 
be permitted to put a ' ?' by any evaluation about which they are unsure, and they will be 
asked to explain why they are unsure. They will be allowed to make borderline ratings 
in exceptional cases. 

ESs will be instructed to rate all articles on the basis of their usefulness, 
regardless of any prior familiarity with them. However, they will be asked to indicate 
which articles they recognize. 

In order to understand more fully the basis on which ESs make judgments 
of the usefulness of documents, we have prepared a list of possible reasons why ESs 
might not judge a document to be "highly useful": 

Insufficient Relevance of Article 

(a) The article is not at all relevant to my research topic, 

(b) The article is only indirectly relevant to my research topic. 

(c) Only a small portion of the article is relevant to my research topic. 

Nature of Article 

(d) The quality of the article is inferior. 

(e) The article's treatment of the topic is too superficial or elementary. 

(f) The article is not fully understandable ; or it lacks textual clarity. 

(g) The article contains no new information , does not move beyond the current 

state of knowledge in”the field. 

(h) The results of the article are outdated. 

(i) The article is experimental , rather than theoretic al. 

(j) The article is theoretical , rather than experimental. 

(k) The orientation of the article is not appropriate (it is overly mathematical, 
is about the wrong material, etc. — please specify). 

Other 

(l) Please Comment 

ESs will be requested to record one or more of the letters, each time they rate a document 
as "somewhat useful" or as "not useful", to explain why the document was not rated as "highly useful." 

The author field will be omitted from the first(main) part of the Series -B ex- 
periments. Three reasons are that author names do not indicate content value, and hence 
are not in themselves good indicators of document usefulness, and author names will un- 
doubtedly be included in standard information-retrieval systems in any event, and will 
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most commonly be utilized by users in conjunction with other catalog information — 
which is the context in which they will be evaluated in Series B. 

Matching subject phrases will appear alone (without title, author, and location). 
As a result of this change we will be better able to draw conclusions about the precise 
value of matching subject phrases per se. 

In the first experimental session, catalog information will not be presented by 
fields, but entries from all fields will be mixed together in random order. The ordering 
of catalog information will help to eliminate biases caused by interest changes of ESs 
which occur over time (fields evaluated first might be affected by variational factors not 
present at the beginning of the field evaluations). It will also eliminate variables which 
may occur as a result of the order in which fields are evaluated. In the second experi- 
mental session the full text will appear without title, abstract, and citation. 

To test the reliability of our experimental procedures, we plan to present the field entries 
and full texts of five documents two times for evaluation during the course of the experiment. 

Summary of Design Changes . We have constructed tighter controls in all 
phases of the experiment in order to improve the validity and reliability of our experi- 
mental techniques. Procedures have been developed to obtain added information from 
the ESs about the nature of their judgments and about the causes of failures of catalog 
information to indicate document usefulness. We have expanded our experimental sample. 
As a result of these changes, we expect to have richer and more extensive data concern- 
ing catalog indicativity; and we will be able to determine with greater confidence the 
operative variables that are influencing the results. 

USER INTERFACE 

Background on Workshop and Paper . A paper describing the user interface of the 
Intrex retrieval system, and our evaluation of that interface was prepared for the "Workshop on the 
User Interface for Interactive Search of Bibliographic Data Bases," held at Palo Alto, 
California, on January 14, and 15, 1971. The workshop was sponsored by the AFIPS Infor- 
mation Systems Committee and our paper will appear in the proceedings. It was one of 
eleven invited papers that provided the basis for the workshop discussion in which two 
Intrex staff members participated. A digest of the paper is presented here. 

Our paper provides a unified, comprehensive description of the current user 
interface and some preliminary results of our evaluation of that interface. The manner 
in which the user interface is implemented has been described in earlier Intrex Semi- 
annual Reports, but in our paper all the pertinent information has been gathered in one 
place. A major portion of the paper is devoted to a summary of our evaluation of that 
interface based on our experiences with system use. A significant portion of that ex- 
perience has resulted from work during the current reporting period. 

General Conclusions . In general, we conclude that the user interface, 
as currently implemented, does a good job of meeting user needs, as evidenced by the 
high degree of user acceptance. Our evaluation is based on: (I) several dozen detailed 
observations of system sessions including pre- and post-session debriefings, and both 

-26- 




29 



computer and human monitoring of the user's interactions; (2) several hundred usages 
of the M.I.T. Engineering-library station which were monitored automatically by the 
computer and seme of which were also monitored by the Intrex Advisers; (3) infor- 
mation gathered in the course of more formal and controlled experiments; (4) obser- 
vations of training sessions for Intrex Advisers. 

The evaluations presented in the paper are carefully qualified to indicate 
that they are only preliminary results and that more experimentation and analysis is 
needed . 

Users appear particularly pleased with the interactive nature of the system, 
its ease of use and its rapid response, particularly when compared with standard library 
techniques. They appear to be able to make good use of the catalog data and the manner 
in which it is presented. They find that the command language is easy to learn, at least 
in its general outlines, and that many of the instructional features provided are helpful. 
Although no single console configuration combines all the features that users want, all 
consoles have met with favorable response. 

We have observed, however, that there are a number of cautions to be given 
to designers of interactive retrieval systems. For example, although the Intrex retrieval 
system seems easy for most users to get started on, they seem to be relatively slow in 
developing mastery of its full capabilities if adequate personal instruction is not pro- 
vided. (A similar observation is often made by librarians with respect to users of the 
traditional library catalog.) We warned designers of other systems not to expect any 
single approach in the instructional material to satisfy all, or even a majority of users 
but we noted that younger users, particularly those with some computer experience, 
have a relatively easier time learning to use the system than do older persons. This 
augurs well for the future. 

Many of the difficulties that our users have encountered appear to arise 
from system features that either do not work well or do not work in the manner de- 
scribed in the documentation. We feel that these difficulties are due, at least in part, 
to the experimental and evolutionary nature of the system. A second source of diffi- 
culty is system complexity. For experimental purposes, we intentionally offer more 
options than one would expect in an operational system, and many users have difficulty 
mastering all, or even a major portion, of the variations offered. An operational sys- 
tem would be more streamlined. Users also seem to require particular instructional 
help before they can make optimal use of the free- vocabulary indexing. Although this 
kind of indexing is, in a sense, easier to understand than controlled- vocabulary indexing, 
and it is one of the things that lets users get going quickly, many users fail to realize 
the importance of strategies such as synonym usage because they are so familiar with 
contro lled-vocabulary indexing that they have come to expect it without further thought. 
The difficulty can also be expected to diminish as the use of free-vocabulary indexing grow more 
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widespread. In this connection, it is interesting to note that a majority of the partici- 
pants in the Workshop favored at least some aspect of free - vocabulary indexing for 
interactive systems. 

In the paper we evaluated four major aspects of the user interface: doc- 
ument representation in the catalog, the command language, the consoles, and the in- 
structional material. 

Document Representation in the Catalog . The catalog- record structure 
was carefully designed to support a variety of retrieval and display functions. The ana- 
lytically structured catalog record was divided into a large number of machine -distin- 
guishable fields and sub-fields on the ground that while distinctions could always be done 
away with, they could be added later only with difficulty. Most users find our distinctions 
finer than they need. In this sense, different data elements serving the same function — 
such as excerpt and abstract — should be combined for users, although internal tags 
differentiating elements may be necessary for bibliographic control. Some data elements 
serve more than one function; these elements need to be separately retrievable at a 
field level and combinable with others to serve a larger function. Present combination 
fields, such as the ones evoked by the OUTPUT command arguments, NORMAL and 
STANDARD are quite popular with users. 

Users find the manner in which the information contained in the catalog 
is displayed quite important but there appears to be little agreement as to what is the 
best format. We do know that users dislike abbreviations and prefer to have the system 
do the decoding of abbreviations for them. They also tend to favor the fields with which 
they are already familiar, a fact we find somewhat disappointing since it makes the evalu- 
ation of novel fields in the augmented catalog difficult. (Our new design for Category-II 
experiments described in the section above is one attempt to solve this problem.) We 
were hard-pressed to find eood names for some of the fields, but most of the names we 
chose cause little, if any, difficulty. 

We note that the utility of the catalog information decreases as the con* 
venience with which full text can be viewed increases. 

Our method of subject indexing appears to work well both for retrieval 
and for judging document utility. The vocabulary used by authors (and hence by our 
indexers) tends to match the search vocabulary of users. The depth of our indexing 
serves to find documents that users might otherwise miss, but this causes some diffi- 
culties at the interface with users who base their judgments of document relevance on 
more limited information (e.g., the title). The use of synonyms in the set of subject 
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The request "output normal" gives the three fields author , title , and location . The 
request "output standard" gives the same three fields but in the standard, condensed 
format of bibliographic citations. 



phrases referring to a single document simplifies the user's task by providing a variety 
of entry points to documents that are "all about" the subject expressed in the user's 
que ry. 

The subject terms are free-standing indicators of content, which makes them 
useful as indicators of the utility of a document to users. Separation of individual phrases 
also makes it possible to give the user the terms that match his subject request. This 
has at least two benefits at the user interface. It provides, via the "output match" re- 
quest, an indication of document content that is sensitive to the user's interests as ex- 
pressed in his subject request and it gives him feedback on the nature of the matching 
algorithm and how well it is working for his purposes. 

Command language . The Intrex command language features a rigid (but 
simple) formal syntax, with ordinary English system words, We originally employed 
some non-English words but we dropped them when we found that users had trouble learn- 
ing them. Some jargon words in the command language and its description still cause 
trouble but we have had difficulties finding replacements that improve the situation. Ir- 
regularities in the language cause users trouble and we have attempted to minimize these. 
Ambiguities, where they arise, also caus j problems for users. For example, users 
seem to have particular difficulty in handling the DOCUMENT command and in under- 
standing the use of the AND commd. (This latter difficulty appears to have been noted 
by a large number of other workshop participants, too. ) The difficulty with the AND is 
that users have trouble deciding whether they are "and-ing" the search terms (they are) 
or merely summing the document lists (which is the Boolean or). The difficulty with 
the DOCUMENT command appears to lie primarily in the fact that users think of it as 
an output command; actually, it is implemented as a search command. 

Other command- language facilities that appear to please users include the 
ability to abbreviate commands and to switch to the "short" mode, which suppresses or 
shortens most of the instructional material presented in the computer responses. 
Punctuation in the command language causes some difficulties because users tend to 
ignore it or treat it carelessly. The fact that no user-originated command can hurt 
the system and that the system recovers from most errors in such a way that the user 
can easily get going again is important. Users appear to want a pointing capability, the 
ability to control stemming and phrase decomposition, the ability to combine arguments 
and commands, an instructional mode of operation, and user control of such stylistic 
matters as output format and character size. We are planning to add most of these 
capabilities when it becomes feasible to do so. 

Consoles and the Physical Environment. Users appear to be pleased with 
all of the consoles provided and with the physical set up in both the Engineering Library 
and the E.S. L. facility. No single console, either provided by us or available on the 
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market, combines all the desirable features, but even the more minimal consoles seem 
satisfactory to users to at least some degree. 

The major advantage of typewriter consoles is that they provide immediate 
hard copy and they are preferred by some users for this reason alone. 

The ARDS console provides more rapid output and silent ope ration but lacks 
hard copy capabilitie s . Users are particularly pleased with our combined configuration 
in which the ARDS screen is also used for full-text display. The rapid access to full 
text, maybe a major factor in making interactive retrieval systems attractive to the 
majority of potential users. 

The Intrex display console has greater clarity, size and brightness in its 
characters and this feature elicits a favorable reaction from many users. It also allows 
users to save and redisplay pages and thus helps to compensate for the lack of hard copy. 
We do not yet have sufficient information to evaluate the extended character set (192 
characters), status lights, programmable buttons, editing features, and other special 
features of this console. 

Instructional Features . We have found that users learn best by actually 
using the system. Here the rapid and informative responses by the system are ex- 
tremely helpful. We also find that no single instructional mo d e pleases all users. 

Users seem to favor clie online instruction even when warned that some of its passages 
may be out of date. They appear to want both a brief introduction and a complete refer- 
ence guide, and we have provided these. 

We have found that it is much more difficult to instruct users than we had 
originally imagined, especially from written manuals or guides. One reason for this 
appears to be intrinsic to the library situation. Users, in general, are much more in- 
terested in getting the information that they are seeking than they are in learning about 
extended use of the system. It is important to provide a simple operational core sys- 
tem, within the more complex, full system, that users can use to "get started." They 
can then proceed to expand their capabilities as they feel the need. It appears to be 
quite easy for most users to get started on Intrex. Both the system dialog and the free- 
vocabulary indexing contribute to this. 

Summary . We summarized our paper by observing that the experience to 
date suggests that the Intrex interface has already achieved a considerable measure of 
success. Our experience also suggests, however, that improvement is possible in sev- 
eral areas. Among these areas are cost reduction and hardware improvement. 



For additional discussion on the capabilities and limitations of online, interactive 
consoles see: Marcus, R.S. and Tewlow, J.S. , "Computer Terminals for News- 
papers: Applications and Requirements," Proceedings of the IEEE Computer 
Group Conference , June, 1970. 
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One observation that we would make as the result of our attendance at the 
Workshop itself is that there appears to be no other group that is nearly as well equipped 
at this time to perform carefully controlled experiments to evaluate augmented catalog- 
ing, deep and/or free -vocabulary indexing, various console features and online text 
access. Intrex was unique among the systems represented at the Workshop and the 
results being obtained from Intrex experiments should be helpful in optimizing future 
retrieval systems . 

MICROFILM READER 

Under sponsorship of the U. S. Office of Education, staff members of the 
Electronic Systems Laboratory have designed and constructed two versions of a micro- 
film reader with features not ordinarily found in commercially available readers. Heavy 
emphasis has been placed on human factors, particularly reader comfort and convenience. 

Two configurations of the reader have been built; they have been dubbed the 
"desk-top version" and the '‘arm-chair version". In both versions a blown-up image of 
the material contained on a frame of microfilm is created on an opaque surface, and 
this image is viewed at a comfortable reading distance by the user. In the desk-top 
configuration, Fig. B-3, the unit is mounted atop a simple desk-like surface and the 
user is seated in a normal reading-writing position at the desk. The unit can be swiv- 
eled or tilted, to accommodate the user’s reading preferences. In the other configura- 
tion, Fig. B-4, the image is projected onto a larger vertical surface and the user is 
seated for reading in an arm chair approximately five feet from the reflecting surface. 
Frames are selected by means of a control box at the arm-chair location. The arm- 
chair reader has been designed to accept 4-inch by 6-inch microfiche; the desk version 
takes the 3 5 -mm film strips generated by the film station developed by the Project 
Intrex staff. 

The goal of the designers is to implement a film- reader system which offers 
the reader a crisp viewing image that closely approximates that of a printed page and a 
reading environment that minimizes eyestrain and body fatigue over prolonged periods 
of reading. In contrast to most existing microfilm readers, the devices now undergoing 
evaluation have essentially nondirectional image characteristics. Hence, the reader 
has neither to sit bolt-upright before a viewing surface, nor must he view the images 
essentially head-on to obtain maximum brightness and discernibility . 

The vertical- screen viewer has been installed in the Barker Engineering Li- 
brary and is being evaluated there through responses to questionnaires given to users. 
The desk-top viewer will be used in conjunction with the Intrex installation being planned 
for the M.I. T. Center for Materials Science and Engineering. 
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Fig. B-3 The desk-top version of the Electronic Systems Laboratory 
microfiche reader. 
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Fig. B-4 The arm-chair version af the Electronic Systems Laboratory 
microfiche reader. 
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SUMMARY 

A change from an off-line paper tape method to a direct on-line method of key- 
boarding catalog- record data has been implemented. This change was necessitated by 
discontinuance of the paper-tape to disk-storage converter facility at the Information 
Processing Center. Other alternative input methods that were examined, the decisions 
effecting implementation, and initial operational data, are discussed. We are also 
using, temporarily, an indirect paper-tape to magnetic-tape conversion method to pro- 
cess our backlog of paper tapes which were generated prior to on-line switchover. 

Paper tapes being created in a temporary back-up mode to the on-line method are also 
passed through the indirect paper-tape to magnetic-tape conversion process. 

We have revised the specifications for three catalog- record fields containing cor- 
porate- and geographic-name elements. We are gathering and interpreting illustrative 
examples of our deep subject- indexing methodology. Section B of this report includes 
a discussion of staff indexer participation in the advisor-training program and a digest 
of the effect on the user interface of Intrex's bibliographic record-design and its access 
points. 

As of 15 March 1971, 17, 350 documents have been indexed and 14, 100 catalog 
records have been completely processed into the computer-stored data base. 

PROCESSING 

General. The input group has initiated a periodic report to the text-access and 
softv/are personnel informing them of the fiche numbers associated with those catalog 
records which are ready each month for inclusion in the data base. This procedure 
will allow the text-access facilities, which are limited to 1500 microfiche, 'tnd the re- 
lated software to be updated to reflect the most recent documents in the Intrex system. 
This item is discussed further in Section F of this report. 

We have introduced an additional step into our correction loop. This loop consists 
of a two-cycle process of manual proofreading followed by on-line editing of catalog re- 
cords. Previously, no further manual proofreading was done after the second-edit step; 
we relied on visual verification that is a part of the edit process. We have since found 



it necessary for a proofreader to check the hard-copy dialog transaction of the second 
editing. This correction procedure is a relatively quick process because of the small 
number of edit transactions at that stage (about 5 per file of 10 documents); it has 
helped to minimize the number of catalog-file rejections by the format proces sing pro- 
gram. 

Our correction loop which ultimately ends with catalog records entering the stored 
data base was only recently resumed after a two-month halt because of hardware prob- 
lems in the high-speed channel of the 7094 computer. At that time, catalog data had to 
pass through that channel on its way from paper-tape storage to disk storage. This pro- 
blem is discussed below in connection with our change to on-line data input. 

As of 15 March 1971, 

17, 350 documents were indexed 

17, 250 catalog records were reviewed 

15,780 records were keyed 

14,740 records were passed through the 

first proofreading and editing process 

14, 100 records were completely processed 
into the computer-stored data base. 

On-line Data Inputting. We have changed from an off-line to an on-line method of 
keyboarding catalog- record data. Previously, catalog records were keyed onto punched 
paper tape with Friden Flexowriters . Each paper tape segment contained ten catalog re- 
cords, called a file. At one- month intervals, the accumulated paper-tape files were 
passed through an optical paper-tape reader connected to a PDP-7 computer, a satellite 
of the central IBM 7094 computer. The PDP-7 then converted the Flexowriter codes to 
ASCII codes and transmitted the data over a high-speed cable to the central computer. 

It is this sequence of events, from initial keyboarding of data to its entry into the central- 
computer disk files that has been converted to direct on-line input. The two major pro- 
cessing steps after this point remain unchanged. These are: correction-loop processing, 
and formatting and merging the new records into the stored data base. 

Change-over to on-line inputting was brought about by the plans of M. I.T. 's In- 
formation Processing Center to abandon their PDP-7. A study by C. W. Therrien of the 
Intrex Software Group showed alternative methods of catalog input and associated pro- 
cessing costs to be more expensive than our regular off-line paper-tape method. How- 
ever, arrangemenrs recently completed with the Information Processing Center permit 
on-line inputting at essentially no incremental cost to our project. 

Three years ago (see the 15 September 1968 Semiannual Activity Report) a cost 
analysis of on-line keyboarding and paper-tape inputting showed the on-line method to 
be twice as expensive as the paper- tape method ($41. 07 versus $23.46 for a file of ten 
catalog records.) The recent study of alternative input methods updated cost figures 
of the earlier study* In real cost, on-line inputting is still twice as expensive as the 
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paper-tape method ($43. 02 versus $23.65 per file of ten catalog records). The double 
cost of on-line input is almost entirely the result of the need for additional computer 
processing. Seventy percent of that excess cost (about $14. 00, or one-third of the total 
on-line cost) is attributable to computer-program swap time encountered within the 
computer while the typist is inputting data. The total on-line keyboarding cost would be 
reduced to $29.20 per file if program swapping were not involved. A configuration 
which buffers six lines of input data is possible to implement and would reduce swap 
time by a factor of si:*', thereby resulting in a total on-line keyboarding cost of about 
$31.50 per file. 

Other major alternatives for input that were considered were paper-tape to 
magnetic-tape conversion and direct key-to-tape methods. The direct key-to-tape 
method was rejected because of incompatible magnetic tape, increased rental cost of 
key-to-tape devices, and the ability of most such devices to display only the last char- 
acter typed, or its bit configuration, rather than the line currently being typed. We 
consider full-line display (and preferably several lines) important for keying our data 
which are mostly variable- length natural English text. 

Paper-tape to magnetic-tape conversion is an indirect but feasible alternative. It 
is indirect because the 9-channel output magnetic tape from the paper-to-magnetic tape 
converter available to us must be processed through an IBM 360/65 computer to pro- 
duce specially- formatted 7-channel tapes compatible with the IBM 7094. In addition 
Flexowriter-to-ASCII code conversion must be done on either the 360/65 or the 7094. 
The cost of paper-tape keyboarding plus conversion to magnetic tape and indirect pro- 
cessing for the 7094 computer is in the range of $26 to $28 per file. The indirectness 
of this data-transfer conversion is more of a disadvantage than its additional cost over 
the previous paper-tape to PDP-7 satellite computer method. 

The plans and protocol to implement daily on-line keyboarding were developed 
over a four-week period and included trial input runs to familiarize typists with this 
mode and to identify operational problems. The difficulties encountered were minimal, 
primarily because of our extensive experience with on-line editing of catalog data. We 
wish to highlight major implementation decisions and experiences. 

On-line data input is implemented as one function of generalized editing. Choice 
of a general text-editing program will affect the general data preparation process. Two 
such editing programs (designated QED and EDA) are available to us through the 7 094 
computer system. Their general features were studied in the total context of editing 
and QED was judged the more powerful for the following reasons: (1) switching from 
a strict input mode (called "append" in QED) to other edit modes, or the reverse, is 
less problematic because QED uses more explicit commands; (2) QED has both for- 
ward and backward line-pointer movement; (3) sev eral s equential-edit commands can 
be combined onto one transaction line in QED. In addition, QED is currently used for 
on-line editing exclusive of data input and one of our two typists was thoroughly familiar 
with it. 
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Two IBM Z741 terminals previously available to all systems personnel are now 
reserved for exclusive use by our two typists. Differences between the Flexowriter 
and Z741 keyboards emphasize the interdependence of data- element coding and de- 
limiting with the keyboard character set and the location of specific character keys. 
Typing experience resolved minor start-up difficulties with the location of equivalent 
keys. However, a few delimiters in the data are somewhat less than optimum now for 
keyboarding purposes either because of their specific location or the necessity for a 
case-mode change. The character- s et differences posed other problems. An escape 
convention is now used to key four infrequently occurring graphic characters. Two fre- 
quently occurring graphics were dropped altogether by revising the specifications for 
three catalog fields; a confusing escape sequence for those particular graphics was 
avoided. 

The transition to an on-line keyboard resulted in easier correction procedures for 
sensed errors. Two reserved characters used throughout the CTSS on-line system for 
"erasing" keyed characters or cancelling a complete line are used. 

Two protective features were built into the protocol and instructions given to the 
typists which would prevent significant data loss in the event of a major computer fail- 
ure. A typist is required to write out newly keyed data into a temporary-save file after 
each catalog record, or more often if a particular record is lengthy or difficult. File- 
name protection incorporates the typist's own name or initial into the names of the 
temporary-save file and completed file. It prevents one typist from accidentally de- 
leting the files of another typist. 

On-line inputting began on a regular basis on 13 January 1971. Start-up proceeded 
smoothly but three weeks of full-time operational experience is insufficient for a de- 
tailed review. We can note, however, that two major benefits to the entire processing 
cycle are now possible. On-line input realistically offers quicker turnaround and a re- 
duction in elapsed processing time. A newly keyed file can be made available for proof- 
reading the morning after it was created. However, we will continue correction- loop 
processing within a monthly cycle until more on-line input operational experience is 
gained. A second benefit is that the full power of the QED general editing program can 
be applied during initial keying so that we could expect cleaner records to enter the 
correction loop. One typist experienced in editing quickly found it advantageous, easy 
and natural to switch modes within QED during initial keyboarding to correct errors she 
detected on lines other than the current input-data line. 

The only serious problem with on-line input in a production environment, other 
than its cost, is computer down time. For our time-sharing system there is exceed- 
ingly little interference between our daily daytime production mode and the regularly 
scheduled preventive-maintenance sessions on the computer; these cut into about 1 l/Z 
hours of our work week. Unscheduled down time caused by computer failures is more 
serious for production because of its unpredictability of occurrence. Its duration 
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depends upon the severity of the problem and may last anywhere from a minute to several 
days. We are currently exploring several potential methods of back-up and work schedul- 
ing during computer failures. 

We are temporarily using our Flexowriters as a back-up mode to on-line input. The 
paper tapes created in this mode, and backlog paper tapes created before the switchover 
to on-line input are being passed through the indirect paper-tape to magnetic tape con- 
version process discussed earlier. 

During our first three weeks of on-line input, 73 percent of all records keyed were 
keyed on-line. For on-line input, the average computer time, measured in terms of time 
consumed in the Central- Proce s sor Unit, expended on input processing was 0.38 minute 
per catalog record. The ratio of computer time to console time was 0. 013. These figures 
are comparable to the data obtained in the on-line input analysis three years ago. 

On-line data input now brings the general data-preparation process closer to "he 
basic on-line nature of the experimental Inirex retrieval system. At some point, it will be 
of interest to do trial input runs at display terminals to investigate their efficiency and 
other effects on production-type work. ARDS terminals and the specially designed Intrex 
console are both available. The Intrex console also offers the potential for localized data 
editing with a cursor and function buttons. 

DATA ELEMENTS AND CODING 

We re-examined and partially revised the specifications for three catalog record 
fields containing corporate and geographic names. The data fields are author's affiliation 
(field 22), corporate author (field 23), and contract statement (field 40). Only the formatting 
and coding of the corporate and geographic names in those fields were affected; specifica- 
tions for other data elements in those fields remain unchanged. The impetus for the 
revision was the impending change to on-line input and the need to avoid a confusing escape 
convention to key geographic name delimiters in those fields. The revision itself had 
broader benefits for both catalogers and typists. A cataloger's work with a cumbersome 
manual authority file of corporate names was reduced to a fraction of the time formerly 
spent, and a typist now has several less keystrokes and key escape conventions to make 
in each record. 

Essentially, geographic names are no longer tagged in all three fields and, except 
for field 23, corporate names are no longer established. Whatever form or degree of 
completeness a corporete name has in the document at hand will be accepted, but the order 
of the organizational elements present will still proceed from largest to smallest to main- 
tain some consistency in their display. Corporate names will cont nue to be established 
for field 23 (corporate author) because the elements from this field bear more directly on 
general bibliographic control and are more likely to be used in citing a document than 
similar elements assigned to the affiliation or contract fields. 

Although there is no current inverted-file index to corporate and geographic names 
in the Intrex catalog, these revisions would not affect the creation of such an index along 
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the same lines as the current subject/title inverted file where access to records is pro- 
vided by post coordination of individual words. 

EVALUATIONS 

At the beginning of this reporting period, the staff indexers initiated work on the 
gathering and interpreting of examples illustrative of the subject-indexing methods em- 
ployed by Project Intrex. This continuing effort is directed toward complete documenta- 
tion of our free- vocabulary, free phrase, in-depth indexing methodology. 

The on-line user interface is influenced in many ways by the design and organization 
of the bibliographic records and the access points to those records. The results we have 
found to date with real system use, as affected by the experimental Intrex catalog-record 
design and its many access points, including deep subject indexing, are discussed in a 
paper prepared for the "Workshop on the User Interface for Interactive Search of Biblio- 
graphic Data Bases". A summary of that paper appears in Section B of this report. 
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SUMMARY 

Extensive documentation of the Intrex software has been initiated. A large ef- 
fort is being devoted to an Intrex report which will describe in detail the on-line re- 
trieval programs and the programs used for data-base generation. In addition three 
internal reports were prepared on the buffer/controller software for the Intrex con- 
sole. These reports will be included in a larger Intrex report to be issued later. 

As the data base continues to grow, it has become expedient to find a means for 
decreasing the cost of adding new documents. A new procedure has been implemented. 

Minor software changes were made in the retrieval system to respond with an in- 
formative message to requests for full text of older documents which, due to limita- 
tions in capacity of the text-access subsystem, have been removed and are retained 
off-line. Several other small software modifications were made to improve system 
performance and correct defects. 

Software for the buffer/controller was modified to accommodate the additional 
4-K words of core that were added to the 620 i computer. The additional core capacity 
makes it possible to provide automatic switching to text- access mode on the In- 
trex console through use of the command "output text". This feature had previously 
existed only on the ARDS combined terminals. 

Finally, a short-term effort was expended to aid a project sponsored by the 
Cambridge Project at the MIT Center for International Studies (CIS). The CIS pro- 
ject involves the indexing and retrieving of political documents. A goal of this ef- 
fort was to develop a word concordance of 150 newspaper editorials as part of a 
larger effort to develop vocabularies suitable for indexing this type of material. Our 
part in the project was more than a success in that we not only developed the desired 
concordance, but also, through Intrex- like retrieval, uncovered relevant editorials 
not previously found by CIS personnel. 

RETRIEVAL SYSTEM SOFTWARE DOCUMENTATION 

The programming staff has devoted most of its time to writing comprehensive 
documentation of all of the software developed by Intrex. When completed, this re- 
port will discuss each of the 300 separate subroutines that comprise the Intrex re- 
trieval system. Also included in this report will be documentation of the programs 
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used for data-base generation. To date, most of that part of the report which describes 
program details has been written. Other smaller portions that deal with an overview of the 
software and general programming- system architecture still remain to be completed. 

RETRIEVAL SYSTEM UPDATE AND MAINTENANCE 

The data base was increased by 2^20 documents to a total of 14, 100. Changes were 
implemented in the data-base generation process which reduced the computer cost as- 
sociated with inputting to $2.65 per document. This is a substantial reduction from the 
previous figure of $3. 50 per document. 

The online directory to the microfiche collection has been updated to correct an ac- 
cumulation of missing entries and errors. 

Since there are now more microfiche than the storage-and-retrieval equipment can 
hold, a scheme has been implemented to give users the best possible service under this 
limitation. The earliest microfiche have been removed to make rcom for more recently 
acquired documents. If a user asks for text which is on a fiche no longer included in the 
online collection, the following message appears: 

"This document is not available at the console. You may obtain a copy of it 
from the Microfilm Service Area at the MIT Engineering Library 
(UN 4-6900, Ext. 3129). " 

Presently, the first one-hundred fiche (out of a total of 1500) have been removed 
to make room for more recent documents. 

Several other software improvements were implemented, as described below: 

1. The online User Guide and dialog files were brought up to date 
and streamlined to remove outdated cr misleading message text 
which had been confusing users. 

2. The command delimiter and trim tables were updated and stan- 
dardized. Thus the free format with regard to punctuation and 
delimiters between parts of a command was extended to include 
most Intrex commands (all except those where special treat- 
ment is needed). 

3. The list-pointer table was expanded in size to allow more lists 
of documents to be named in a single user session. The cur- 
rent limit is now 38. 

4. A central-error exit was included which takes control after most 
types of system errors (except "program stops"). This tech- 
nique provides more informative error messages to be given be- 
fore allowing the Intrex subsystem to take control. The core- 
image of an abortive session is saved and given a special name by 
the subsystem, thus allowing postmortems to be performed. 

5. More efficient use of a portion of computer core known as "free 
storage" was provided by removing the command line buffer from 
free storage and placing it in a separate region of core. 
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6. Recognition of the QUIT command while in the “login” stage was 
enabled, thus offering to the user an opportunity for an imme- 
diate exit from the system during the login. 

7. As a result of the close perusal given the various sections of soft- 
ware during documentation, many redundant or obsolete para- 
meters and pieces of program code were eliminated. 

In addition, a few new bugs were discovered in the retrieval system; these have 
been corrected. The extensive use that the system received at the hands of real users 
this fall and winter enabled the Intrex programmers to identify problems that had eluded 
detection in previous testing. 

BUFF ER/CONT ROLLER SOFTWARE 

The buffer/controller software has been modified to take advantage of the ex- 
pansion of the 620i core memory from 4-K to8-K words. This expansion made itpos- 
sible to eliminate the core/drum overlay system forced on the 620i monitor software by 
inadequate core storage. Because of timing requirements, the overlay system was not 
workable with multiple Intrex CRT display consoles and its removal was undertaken in 
preparation for the second Intrex console currently being fabricated. 

Elimination of the overlay system allowed the 620i software to be modified so that 
the automatic text-access mode of the Intrex console could behave like that of the com- 
bined ARDS/Text-Acce s s display console; that is, the Intrex command "output text" 
now causes an automatic access to the full-text subsystem resulting in full-text display 
on the associated Tektronix 6 1 1 storage tube. The user may then continue using text- 
display function switches or return to catalog searching by typing the normal Intrex 
commands on the keyboard. No manual mode-changing is required. The Intrex console 
does, however, retain a text-access stand-alone mode for use when CTSS is not avail- 
able and document fiche numbers are known to the user. 

As a part of system software/hardware documentation efforts, the following in- 
ternal reports were produced: 

(1) A general system component and configuration description of the 
Intrex display system. 

(2) A general functional description of the buffer/controller 620i 
software . 

(31 A principles - of-operation manual for the magnetic -drum unit 
of the Intrex buffer/controller. 

The latter manual contains detailed information and instruction useful to system 
programmers in programming the local storage-drum operations associated with the 
Intrex display consoles. 

INDEXING AND RETRIEVAL OF POLITICAL DOCUMENTS 

The versatility of the Intrex software system was demonstrated through assistance 
rendered to the staff of a project administered at the MIT Center for International 
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Studies (CIS) by the Cambridge Project. The stated purpose of the task brought to us 
was "the development of computer-based methods for identifying themes in political 
documents". This work is noteworthy because it highlights the ability of Intrex tech- 
niques to assist in the retrieval of documented information not specifically within the 
Intrex experimental scope. 

A goal of the CIS project was to develop a word concordance of 150 newspaper 
editorials as part of a larger effort to investigate appropriate vocabularies for indexing 
this type of material. The Intrex stemming algorithm was considered especially appro- 
priate as a means for bringing together morphologically related words . Catalog and 
text records for these editorials were converted to Intrex format with each paragraph 
being tak^n as a subject term. These reformatted records were then run through the 
regular Intrex data-bas e -generation programs. The inverted file thus generated pro- 
vided the desired concordance. 

As a by-product of this effort it was possible to use the Intrex retrieval programs 
to search for editorials on particular subjects. Prior to the completion of the Intrex 
processing, o list of topics had been prepared, and a knowledgable person manually 
searched the text of the editorials to find those related to each topic. Intrex retrieval 
based on keywords from some of the topic statements was then tried. Most of the 
manually found editorials were retrieved along with only a few nonrelevant editorials. 

In addition there was a "discovery set", that is, a few editorials not found manually 
were retrieved by Intrex, and these were judged relevant by CIS pers onnel. Although 
in no sense statistically significant, the results are, perhaps, another indication of 
the efficacy of free- vocabulary indexing. 

The principal investigators on the CIS project are Professors William E. Griffith 
and Rosemarie Rogers. 



Previous work in the use of Intrex techniques for newspaper-story retrieval has been 
reported in a paper by J. F. Reintjes and R.S. Marcus, Computer-Aided Processing 
of the News, Proceedings o f the Spring Joint Computer Conference, 2969. 
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E. DISPLAY CONSOLES 



Staff Members 



Graduate Students 



Mr. J. Bosco 
Mr. J. E. Kehr 



Mr. R. Goldschmidt 
Mr. Y. Chan 



Mr. D. R. Knudson 
Professor J,K. F.oberge 

SUMMARY 

The original design concept for the Lit rex display consoles envisioned several 
cathode - ray- tube (CRT) display terminals interfaced to the main computer by means 
of a local buffer/controller that includes a small computer and a magnetic-drum mem- 
ory. To date only one Intrex display console has be~n operating. Now a second termi- 
nal has been assembled and testing of it is well underway. 

THE SECOND CONSOLE 

The basic design of the first and second terminals is almost identical. However, 
improved integrated circuits have reduced the number of components and allowed more 
efficient packaging for the second display. A photograph of the new terminal is shown 
in Fig. E- 1 . 

The new terminal consists of a CRT and keyboard, which can be placed on any 
available table or desk, with the electronics and logic circuitry located in a separate 
cabinet. The terminal was packaged so that it could be easily moved and also to facili- 
tate maintenance. Other noteworthy features include a CRT with a non- glare faceplate 
for increased contrast and a solid-state keyboard which is expected to be more reliable 
than the photo-electric keyboard on the first terminal. 

The use of more than one Intrex display terminal requires certain hardware and 
software modifications to the buffer/controller* The major hardware modification is 
the replacement of the present drum memory with one of increased capacity and sep- 
arate read-write heads for each of the two display terminals, A Vermont Research 
Corporation 128-track drum with three sets of read-write heads will replace the current 
32 -track drum and is currently being installed. The buffer/controller software modi- 
fications are being designed and will be described in the next Semiannual Report. 

Mr. Y. Chan, graduate student, is investigating methods for communicating be- 
tween the display and the buffer/controller when separation distances are 2000 feet or 
more. Data are transmitted in both directions as bipolar pulses at approximately a 
1 - Megabit-per-second rate. This inves ligation is expected to result in a design for 
full-duplex transmission over a single coaxial table with repeaters located at intervals 
as required. 



Fig. E-l A second version of the Intrex display console* 
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F. FULL-TEXT STORAGE AND RETRIEVAL 



Staff Members 



Mr. P. Campoli 
Mr. J. E. Kehr 
Mr. D. R. Knuds on 
Professor J.K. Roberge 



SUMMARY 

The Advanced Remote Display Station (ARDS) catalog display and the soft-copy 
text display in the Engineering Library, which previously existed as separate terminals, 
have been merged into a combined ARDS/text-access unit similar to that described in 
the 15 September 197 0 Semiannual Report. Experience gained from the daily operation 
of this combined terminal has resulted in several hardware modifications. The mag- 
nify and mode controls employed during text displays have been simplified to facilitate 
the display operation, particularly by those unfamiliar with the terminal. 

The image quality of both the storage-tube displays and the film-copy terminal 
has been improved and made more stable by modifications to the scanner electronics. 

The need for improved reliability has led to an extensive overhaul of the storage- 
and- ret rieval devices by a representative of the manufacturer. 

The Intrex document collection now exceeds the 1500-fiche storage capacity of the 
text-access system. Procedures, have been adopted that provide automatic access to a 
1500-fiche subset which include the most recent documents, and manual access to the 
remainder of the collection. 

A storage-tube display is being constructed to provide text-access at the second 
Intrex console. Text requests from this display, as from the text display colocated 
with the first Intrex console, are transmitted via the display buffer/controller to the 
text-access central station. The design has been modified to make the display operation 
more consistent with the combined ARDS/Text terminals. 

MAGNIFY MODE 

The magnify option provides a 2X enlargement of a quarter-page sector of full 
text. In the initial design three separate push button actions wore required to ac- 
complish enlargement. The user initiated the magnify mode with the DISPLAY SECTOR 
button, selected 1 of 9 sector buttons, and transmitted the request by actuating the SEND 
button. To simplify matters, this sequence has been replaced by a single push button 
action where activating any one of the nine SECTOR buttons will transmit a request to 
magnify that sector. The SECTOR push buttons are lighted whenever the terminal is in 
the text mode and ready to receive a new request. 
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MODE CONTROL 



The function of the mode-control push button on the combined ARDS/Text terminals 
has been changed. Previously, this, push button was used as a manual override of the 
logic which switched between the catalog and text modes and permitted operation as a 
stand-alone text display independently of the computer. In this mode the computer- 
generated fiche and frame numbers are replaced by numbers from the control-panel 
selector switches. The stand-along mode has proved useful for system testing, but 
normally is not employed by the user. Therefore, the stand-alone mode is now con- 
trolled by a switch in the electronics rack and is not accessible to the user, ar.d the 
mode push button is being utilized for a somewhat different function. 

As modified, the mode push button is lighted whenever the terminal is in the ARDS 
(or catalog) mode. Pushing the button causes the terminal to switch to the text mode 
and turns off the light. The terminal returns to the ARDS mode whenever a keyboard 
action is initiated or a computer message is received. The intent of this modification 
is to allow the user to switch manually between catalog and text modes without losing 
the fiche and frame numbers of the last text request. 

MICROFICHE STORAGE 

The Intrex document collection has recently exceeded the 1 500- microfiche 
storage capacity of the text -access system. As a result, a new procedure combining 
manual and automatic means has been implemented for providing access to the com- 
plete set of documents, A 1500-fiche subset is stored in the automatic storage-and- 
retrieval devices and updated periodically to include the higher numbered fiche, thus 
providing automatic access to most recent documents. Duplicate fiche containing the 
early documents removed from the store can be obtained through the manually accessed 
file in the Engineering Library. 

A spare set of microfiche and coded clips has been procured and are stored near 
the text- access retrieval units for quick replacement of damaged fiche. 

SYSTEM PERFORMANCE 

The daily operation of the Engineering Library terminal and information obtained 
from the user experiments have emphasized the need for improvements in the follow- 
ing areas of the full-text storage-and- retrieval system: 

Stability of the image-intensity control 
Image resolution 
Retrieval-system reliability 
Image centering 

Both image-intensity control and resolution have been improved by a modification 
to the video-amplifier circuit. As described in the 15 March 197 0 Semiannual Report, 



the text-access system utilizes two microfiche retrieval units, each containing a photo- 
multiplier tube (PMT) and video preamplifier. Differences between the two pre- 
amplifiers caused variation of the video bias-level control when alternating between 
the two preamplifiers. A circuit modification has been made which allows both PMT's 
to share a common preamplifier. Through careful selection of components, the gains 
and biases from both PMT's can be matched. This produces a more stable image in- 
tensity at the remote terminals and eliminates the need for frequent adjustment by the 
user. 

Until recently, the scanner lens in one retrieval unit was set at an aperture of 
f/5.6 whereas the second unit's lens was set at f/4.0 to compensate for a lower PMT 
gain. After adjustment of the circuit components to match the PMT gains, both lenses 
were set at f/5.6 in order to provide optimum resolution and improved depth of field. 

Further investigation of the 35-mm film-terminal characteristics showed that its 
video amplifier could be eliminated provided a compensating adjustment is made in 
the transmitter video gain. Because the storage-tube screens are essentially 2-level 
bright-dark devices, they are not affected by small changes in the video level. There- 
fore, the film-terminal amplifier was removed and the transmitter gain was adjusted 
to optimize film-terminal performance with a noticeable improvement in the image 
quality of the 35-mm film copies. 

Many of the text-access malfunctions have occurred because of the microfiche 
storage-and- retrieval units. In order to improve system reliability, these units were 
recently overhauled by a representative from Image Systems, Inc. and critical mecha- 
nisms wore realigned using special fixtures designed for that purpose. Several com- 
ponents have been redesigned since these units were acquired; replacement of some 
obsolete assemblies is being considered as a means for further improvement in system 
ruliabilit y. 

The centering of page images on the display is a continuing problem. The page 
locations from one ffche to another currently vary up to 20 percent of the page dimen- 
sions and are sometimes skewed. The closed-loop edge finding technique, described 
in the 15 September 1968 Semiannual Report, corrects for horizontal variations, but 
vertical errors and skewing cause poor page alignment on the display. As a result of 
discussions with the MIT Microreproduction Laboratory, where the microfiche are 
produced, it was decided that most of the vertical variation and skewing occurs ih 
making the contact-print duplicate from the master fiche. Revised techniques will be 
used to maintain the vertical position to within 0. 025 inch of the nominal location and 
skewing to less than 0. 025 inch away from the vertical. These tolerances represent 
less than 5 percent of the vertical-page size, and are expected to provide adequate page- 
centering on the display. 
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The push button control panel on the full-text display is located below the storage- 
tube screen. It consists of the following controls: FIRST PAGE, NEXT PAGE, 
PREVIOUS PAGE, SAME PAGE, LAST PAGE, SECTORS (1-9), and INTENSITY. In- 
dicator lights are provided for SYSTEM DOWN, ERROR, and FiCHE-NO?' 'OUND. 
The only new feature is the LAST PAGE pushbutton which allows the user to request the 
last page of a document by means of a single pushbutton action. This is expected to be 
useful for rapid access to reference lists at the ends of documents. 
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G. COST ANALYSIS 



Staff Members 

Dr. C. W. Therrien 
Mr. J. E. Kehr 
Proies sor J. F. Reintjes 

SUMMARY 

A start has b«en made on a cost analysis for operational Intrex-like retrieval 
systems. A preliminary study described a basic computer structure and estimated basic 
costs for a large-scale system consisting of a number of geographically distributed but 
otherwise identical centers of information retrieval. Follow-up research yielded data 
that supported some conclusions of the preliminary study and enabled refinement of 
others . 

As part of the research topic, visits ere made to two facilities that provide in- 
formation-retrieval services on a relatively large scale. Because of similarities that 
exist between these systems and Intrex, we were able to obtain useful information in 
several areas that could apply to a large scale Intrex-like retrieval system. 

COST-ANALYSIS STUDIES 

A start has been made on a cost analysis for an operational Intrex-like retrieval 
system. Attention is first being focused on the catalog-retrieval part of the system. 

An initial configuration included a number of identical geographically distributed in- 
formation retrieval centers . Each center would contain a computer in the $2 million 
price range, would provide an on-line collection of one million documents, and would 
service about thirty users simultaneously. No direct intercenter communication is 
postulated. Various data bases would be available on a scheduled basis. During non- 
peak hours the computer would be used for other tasks, not necessarily related to in- 
formation retrieval, on a batch processing basis. 

Subsequently, additional inputs were obtained that influence our cons ideration of the 
above system. First, review of some typical indexing statistics confirmed that one 
million documents may not be sufficient in some situations to service a large class of 
users. The Chemical Abstracts abstracting service reviews about 250, 000 new jour- 
nal articles each year, mostly in the field of chemistry. A catalog of one-million docu- 
ments would thus suffice for a comprehensive coverage of the one field of chemistry for 
a period of about four years, but would not take care of others in a community of heter- 
ogenous interests. A data base of one- million documents, however, seems like a 
reasonable estimate of the number of documents that can be available on-line with 
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current technology. New technology in areas such as optical-card storage may, of 
course, increase the feasible size of the data base. All of these considerations confirm 
that centers of information must have a segmented data base, segments of which are 
either: 1) duplicated for each center and available on a scheduled basis or 2) distrib- 

uted geographically and made continuously available to each center by wire or microwave 
communication links. The design of a multicenter system with inter-center communica- 
tion is currently being investigated both from technical and economic viewpoints. 

A second point that arose from a close look at modern computing systems is that 
the performance/price ratio is higher than was previously expected. In particular it 
was found that one could purchase hardware suitable for on-line retrieval purposes for 
an amount between $1 million and $1.5 million. Such a configuration could service 
thirty or more simultaneous users and provide services for other time- sharing, batch 
processing, and real time operations during off-peak hours. With modular increases 
in hardware, a typical system can be upgraded to service up to sixty-four or more 
users simultaneously. A good estimate for the size of the community of users that such 
a system would serve is yet to be determined. The relation of system size to number 
of potential users in a community is very complex since it depends, among other things, 
on the number and location of the terminals, the size of the data base, and the mixture 
of interests in the community of users. As a very rough estimate, however, one might 
assume that each user requires 24 hours of service per year (two hours per month). If 
a system that can accommodate thirty simultaneous users were available for eight hours 
each day, the size of the community that it could service would be about 2600 persons. 
For systems with 64 and 128 simultaneous users, the size of the community would be 
5500 and 1 1, 000 respectively. The information available to the community must, of 
course, be consonant with user-community interests. This implies that unless all mem- 
bers of the community have interest in one field (such as chemistry), a number of differ- 
ent data bases must be made available, perhaps on a time-scheduled basis. 

As a part of our study, visits were made to two facilities which operate retrieval 
systems with a much larger data base and total number of users than Intrex. The 
facilities visited were the National Library of Medicine, Bethesda, Maryland and 
National. Aeronautics and Space Adminis tration, Washington, D.C. The purpose of our 
visits was to acquire some detailed information about the structure of thes e "operational" 
retrieval systems and, if possible, obtain cost data for the various parts of the systems. 

The two facilities visited are similar to Intrex and to each other in several ways. 
The retrieval programs are used on-line as part of a. larger computer system. Con- 
trol of the programs is via a command language, and many similar functions are 
* 

Large files systems such as the IBM 3330 disk or Control Data 821 data file (disk) 
could accommodate a catalog of 1 million documents if the size of the catalog re- 
cord is limited to 600 characters per entry. 
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available to the \iser in each system. Document retrieval is by means of an inverted 
file, although neither of the systems visited uses a free vocabulary to construct the file. 
Because of the similarities several useful items of information were acquired that 
could apply to a large-scale Intrex-like system. In particular, we obtained data on 
communication costs for remote terminals, statistics on indexing and the catalog-input 
process, and information about general system configuration. One of the systems had 
its computer very nearly dedicated to information retrieval and we were able to appre- 
ciate some of the advantages and disadvantages of their particular system architecture 
on the IBM 360/50. Neither of the systems visited had on-line text access so no in- 
formation was obtainable about that aspect of the retrieval process. 
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III. 



MODEL LIBRARY PROJECT 



A. STATUS OF THE PROJECT 



Mr. C. H. Stevens 

The fall semester has given Model Library Project personnel their first oppor- 
tunity to test some techniques for assisting users with traditional library facilities and 
with those that are the result of the new technology. The results of those tests have 
been generally encouraging. 

Response to the availability of point-of-use instructional aids has been good. 
Users voluntarily try these devices and, from the comments they have given us, we can 
infer that minor changes to the scripts and slides will make the instructions given even 
more valuable. Acceptance of Library Pathfinders has been excellent both at M.I. T. and 
at other schools. A cooperative program is emerging that could assist re se arch libraries 
at colleges and universities and in government and industry as well. User preference 
studies for fiche versus hard copy have not produced significant results; there has been 
too little use of the microfiche collection to permit us to observe trends. Steps are 
underway to improve the collection, provide more fiche readers, and advertise avail- 
ability of fiche in order to obtain a level of usage commensurate with the costs of oper- 
ation and of the studies we wish to make. 

J.t is our desire to have other librarians see Project Intrex and the Model Li- 
brary Project in the Barker Engineering Library and to benefit from their reactions to 
what they observe. We are beginning to invite small groups of librarians to visit the 
Barker Engineering Library this spring and summer for this purpose. 



B. POINT-OF-USE INSTRUCTION 



Mr. C. H. Stevens 
Miss M. P. Canfield 
Mr. J. J. Gardner 

The point-of-use instruction program is in the stage of initial user tests within 
the Barker Engineering Library while production of additional units continues. The pro- 
grams currently in use have been placed in close proximity to the reference tools which 
they discuss and notebooks have been Drovided for user reactions to each program. As 
we place additional programs in the library, using varying formats and content approaches, 
we hope to be able to discover the positive and negative aspects of each. There are cur- 
rently four programs in actual operation; two awaiting equipment delivery; two for which 
scripts are being prepared; and one in an exploratory stage. 
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The introduction to the Barker Engineering Library's subject card catalog 
consists of a synchronized sound-filmstrip (running time 2 minutes and 50 seconds) in- 
stalled in a LaBelle Sentinel sound -film strip projector (Fig. Ill - 1 ) - The unit is outfitted 
with standard monaural headphone; the controls are pre-set; and the program is activated 
by depressing a single button. The machine shuts down automatically at the program's 
conclusion. The controls have been placed behind a steel plate and the projector is locked 
to avoid tampering. 

The program introducing the Barker Engineering Library's author-title card 
catalog is also a synchronized sound-filmstrip and the equipment is the same as that used 
for the subject catalog (Fig. III-2). The program runs for 3 minutes and 30 seconds. 




Fig. III-1 LabeMe Sentinel sound-filmstrip projector used for subject card 
catalog instruction program in Barker Engineering Library. 




Fig. 111-2 Labelle Sentinel sound-filmstrip projector used tor author-title card 
catalog instruction program in Barker Engineering Library. 
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The third program which has been in use within the library for an extended 
period is a five-minute introduction to NASA STAR , It is a synchronized sound-slide 
presentation, a format which has the rather considerable advantage over filmstrip of 
being easy to edit and update. 

This presentation utilizes a projection unit designed and fabricated by the 
Model Library staff and Laurence Associates of Boston. The design requirement was 
that it combine a rear projection screen, carousel slide projector, and synchronized 
audio playback unit in a cabinet small enough to fit onto existing shelves and tables in 
the Barker Engineering Library. The present unit meets this design requirement, 
utilizing a Kodak Carousel 850 Projector, LaBelle Plamatic audio unit, front-silvered 
mirror, and 5" x7" rear projection screen, (Figs. HI-3 and IH-4). A companion unit 
is currently being redesigned by the M.I.T. Audio-Visual Department into an even 
smaller unit. 




Fig. ill-3 Sound-slide projection unit used for NASA STAR 
instruction program in Barker Engineering Library. 




Fig. 1 1 1-4 Interior view of sound-slide projection unit. Clockwise from 

hand set the components are: Kodak 850 Carousel slide projector, 
Labelle Plamatic audio unit, silvered front mirror and 5“ x 7" 
rear-projection screen. 
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The program automatically begins when a hand phone is lifted from its cradle 
and shuts down automatically at the program's conclusion - ready for the next user. 

A presentation on Science Abstracts: Series B, Electrical and Electronics 
Abstracts is in 1 6 mm sound-filmstrip format and will be shown via a LaBelle Courier 
16. This unit is smaller than the Sentinel, will easily fit on a standard library book- 
shelf, and will be activated by depressing a single button. Delivery of this unit is ex- 
pected by 1 April 1971. 

A three-minute introduction to Engineering Index is in synchronized sound- 
slide format and will be installed in the unit currently being redesigned by M.I. T-'s 
Audio-Visual Department. This unit will include a LaBelle Plam?tic for audio and will 
incorporate a Buhl right angle lens in the Kodak Carousel projector. The use of the 
Buhl lens will decrease the size of the unit, making it more readily adaptable to typi- 
cally crowded library environments. 

In an effort to determine user preference between audio-visual and audio for- 
mat we will test two audio presentations. 

An audio program instructing the listener in the use of NASA STAR has been 
completed and placed with NASA STAR in M.I.T 's Aeronautics and Astronautics Li- 
brary, a branch of the Barker Engineering Library. The program runs for approxi- 
mately four minutes and refers to copies of sample pages from STAR contained in a 
notebook adjacent to the unit. The equipment was designed and fabricated by M.I.T. ’s 
Audio-Visual Department and consists of a cabinet containing a SONY cassette player/ 
recorder utilizing a newly available continuous loop cassette. The listener lifts a 
Cousino hand phone off its cradle to activate the program and it shuts down automatically 
at its conclusion. The unit measures 12 n deep x 10" wide x 12" high. (Figure III-5) 




Fig. It 1-5 Audio unit used for NASA STAR program in the Aeronautics and 
Astronautics branch of the Barker Engineering Library. 




-56- 



59 




An audio presentation is also being prepared for Science Citation Index . It 
will be presented in equipment identical to that used for the NASA STAR program and 
will be tested in the Barker Engineering Library. 

The remaining program currently being produced is a synchronized sound- 
slide introduction to the Intrex console. This is being done at the suggestion of several 
Barker Engineering Library users and wi.ll be a brief program intended to overcome the 
more common misconceptions about the Intrex augmented catalog and text-access sys- 
tems. The program will instruct the user on how to get into the system and how to do 
simple searches. Our judgment is that complex or in-depth questions about the system 
are best answered on a personal, individualized basis and through the experience of 
actually using the system. 

In dealing with library users' specific problems in utilizing specific library 
reference tools, we have realized that we are approaching a second-level problem. The 
initial or first- level problem is to make the library user aware of the existence of 
these tools. A point-of-use introduction to NASA STAR is helpful to a user only if he 
knows that STAB exists as a potential source of information. In an attempt to approach 
this problem, we are exploring the possibility of producing i videotape introduction to 
the Barker Engineering Library indicating, as briefly as possible, the various kinds of 
information sources and services available. A decision on whether to proceed with this 
project will be made in the coming weeks. 

The user response to the three programs which have been operational for a 
significant length of time has been, for the most part, favorable. Comments are made — 
at the user's option — in a notebook placed with each unit and replies to each comment 
are typed on the opposite facing page. Additional reactions have been offered by the staff 
of the Barker Engineering Library. The reactions have indicated some interesting pit- 
falls which we had not anticipated and which will help us not only in revising the existing 
programs but also in planning new programs. 

The comments on the subject catalog program fall into these six categories: 



1 . 


Uncritically favorable 


13 


31% 


2. 


Favorable with reservations 


11 


26% 


3. 


Favorable to the concept 
but unfavorable to specific 


10 


24% 




program 






4. 


Totally unfavorable 


2 


5% 


5. 


Irrelevant 


3 


7% 


6 . 


Equipment problems 


3 


7% 



The comments falling into catagories 2 and 3 total 21. Of these, 12 indicated 
the program was too elementary — that the user knew everything or almost everything 
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that was presented. Seven suggested adding more information and two were bothered by 
the inclusion of non-instructional, 'light" material. Some typical comments from these 
categories were: 



"Enlarge it to include more information. 

Good start. Thank you. " 

"Now that I've heard it, I know that I 
shouldn’t have bothered — it's all old 
hat (to me anyway). " 

"This is a good approach to teaching 
library skills, but I think the material 
presented is already common knowledge 
to college students or professors. " 

"Great idea — a bit elementary in approach — 
perhaps it should elaborate more on location 
within library — i. e. , where to find. . . " 

"Not bad. Kind of elementary, but I 
like talking to pretty librarians better. " 

"Can't see how anyone could be here 
and not know what you tell. " 

Of the two comments which were totally unfavorable, one complained that the 
machine was "cold and metallic," the other that it was an "inane waste of money." 

The equipment problems category reflects some minor synchronization prob- 
lems we had immediately after the unit was installed. Since that period of a few days 
the unit has been reliable, requiring virtually no service. 

The category breakdown of the comments on the author-title catalog program 
was as follows: 



1 . 


Uncritically favorable 


12 


34% 


2. 


Favorable with reservations 


12 


34% 


3. 


Favorable to the concept 
but unfavorable to specific 


1 


3% 




program 






4. 


Totally unfavorable 


3 


9% 


5. 


Irrelevant 


2 


6% 


6 . 


Equipment problems 


5 


14% 



Cf the 13 comments within categories 2 and 3; six suggested adding additional 
information; five were related to environment (e.g., need for stool to sit on); one indi- 
cated the program was too elementary; and one found the non-instructive introductory 
material disconcerting. 
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Comments included: 



"Good, but don't lay the wit on so heavy. " 

"It was good but you need a place to sit." 

"Too fast." 

Good idea, but how about one which overviews 
the whole library." 

The three totally unfavorable comments were that "real people could do a 
much better job;" that it was "too expensive;" and "this is inane." 

The equipment problems were similar to those encountered in the subject 
catalog program, and like those, were resolved a few days after installation. 

The comments on the NA SA STAR sound- slide program are categorized as 

follows : 



1 . 


Uncritically favorable 


5 


21% 


2. 


Favorable with reservations 


10 


42% 


3. 


Favorable to the concept 
unfavorable to specific 


0 


0% 




program 






4. 


Totally unfavorable 


5 


21% 


5. 


Irrelevant 


1 


4% 


6 . 


Equipment problems 


3 


12% 



Of the ten comments in category two: seven thought the program went too fast 
two would like to have had sample issues integrated with the program; and one thought 
the screen illumination was too bright. 

Comments included in category two were: 

"Very well done. Indexes should be at hand. I am 
one of that minority who would rather have a written 
guide that I could read at my own pace and order," 

"Good but would prefer a slightly longer time 
stop on pictures." 

"Very good a little too fast in some parts." 

"In general, pretty good and clear. At times, 
slides change too quickly." 

Of the five totally unfavorable comments, four were directed towards the in- 
troduction to the program which is as follows: 
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Library users have always been forced to 
play by the establishments rules. The rules 
for Scientific and Technical Aerospace Reports, 
known as STAR , have been made by NASA 
and are complicated enough to make you 
question the viability of our institutions. 

But — power to the people — that's what 
this three -minute tape will give by explaining 
how to use STAR. Either this or anarchy in 
the stacks. So, Right OnJ 

The four comments indicated a negative reaction to our use of political rhet- 
oric. Obviously these users took the language more seriously than we had intended. 

The programs for Science Abstract. Series B, Electrical and Electronics 
Abstracts and Engineering Index were delayed for both audio and visual editing, pri- 
marily in the introductory or non- instructive areas. These changes were made at the 
request of staff members of the Barker Engineering Library and revolved around anal- 
ogies considered offensive to women. The introduction to Science Abstracts made an 
analogy between using Science Abstracts and taking a drive with a woman driver. The 
Engineering Index script (reprinted in the last semiannual report) compared locating 
correct subject headings to being at a "college mixer, sifting through a multitude of 
entries to find the few chicks, or even the one chick, with the best potential. " 

The criticism of the NASA STAR ; Science Abstracts and Engineering Index 
scripts indicates a need for sensitivity to the current social and political movements. 

This seems especially true at a university, but is probably an important factor in any 
environment. We included light, ir ‘elevant material in the initial programs in the hope 
it would relax the user and add life to potentially dull subjects. This approach may be 
valid, but avoidance of controversial subjects seems advisable. 

The audio NASA STAR program has not been in use long enough for us to re- 
ceive significant feedback. 

In addition to the revision of the Science Abstracts and Engineering Index 
scripts, the comments on the programs have directed us towards considering revising 
the NASA STAR audio-visual program and increasing the time allowed for viewing se- 
lected slides . 

The problem of helping users to avoid viewing the catalog presentations when 
they already know how to use the catalogs is being approached by indicating explicitly on 
signs at the projectors that the material is introductory and directed towards the new- 
comer to the library. We feel that this is the person who can be most helped by this kind 
of presentation and that individualized, complex ^problems in using the card catalogs are 
best solved by librarians. The range that these roblems can fall within is so large that 
a presentation covering them all would be prohibitively long. 

The lemaining programs to be completed are being done in a straight-forward, 
business-like manner, eschewing attempts at humor or lightness. We will try to determine 
if users actually prefer this to the lighter presentations. 
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With the assistance of Mrs. Susan Brown, psychologist, we are developing 
questionnaires based on the comments we have received in the notebooks. We will mea- 
sure reactions of \isers to the equipment, the tone of the productions, the content of the 
productions, the timing of the productions, and the concept of point-of-use instruction. 

We will also determine whether users are viewing the programs out of actual need or 
curiosity and whether these factors affect their opinions of the programs. 

We will continue to share our work through the loan of programs to interested 
institutions. We will offer copies of the catalog programs in slide and 1/4" audio tape 
format, and we will offer both sound- slide and sound-filmstrip programs for NASA STAR , 
Science Abstracts: Section B ., and Engineering Index . We will also loan copies r*f the 
audio NASA STAR program. As additional programs are completed we will continue to 
provide copies for loan. 

We also plan to offer copies of the programs, in various formats, for perma- 
nent retention. Charges made would be to cover our duplication costs only. 
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C. PATHFINDERS 



Mr. C H. Stevens ! 

Miss M. P. Canfield \ 

Mr. J. J. Gardner j 

Mrs. E. King J 

Miss K. C. Todd '! 



In addition to the eighty-two Pathfinders listed in the last semiannual report, 
a total of forty-five have been completed dealing with topics in civil engineering, com- 
puter technology, electrical engineering, humanities and social sciences, information 
science, materials science and engineering, and pollution. Placement of Pathfinder titles 
within these categories is arbitrary and based on local considerations. The titles are: 



Civil Engineering 



Information Science 



Asphalt 

Plates 

Portland Cement Concrete 
Shells 

Soil Cements 

Systems Analysis 

Urban Mass Transportation 



Information Retrieval 
Info rmation T ransmis s ion 
Scientific Information Transfer 



Materials Science and Engineering 



Computer Technology 

Magnetic Disk/ Drum Storage 
Magnetic Tape Drive 
Magnetic Tape Storage 



Ferroelectrics 

Ferromagnetism 

Fiber Composite Materials 

Mossbauer Effect 

Particle Optics 

Raman Effect 

Supe r conductivity 



Electrical Engineering 



Integrated Circuits 
Lasers /Masers 
Magnetohydrodynamic s 
Microwaves 

Stroboscopic Photography 
Telecommunication 



Humanities and Social Sciences 



Black American Novels-20th Century 
Encaustic Painting 
Etruscan Language 
History of the English Bible 
New England Transcendentalism 
Sleep Deprivation (1960-1970) 



Pollution 



Air Pollution - Atmospheric Monitoring 

Air Pollution - Automotive Exhaust Emissions 

Air Pollution - Electrostatic Precipitators 

Air Pollution - Filtration 

Air Pollution - Nitrogen Oxides 

Air Pollution - Standards/ Legislation - U.S. 

Wastewater Treatment » 

Activated Sludge Process 
Wastewater Treatment - Electrodialysis 
Wastewater Treatment - Foam Fractionation 
Wastewater Treatment - Sedimentation 
Water Pollution. - Mercury 
Water Pollution - Monitoring Water Quality 
Water Pollution - Radioactive Materials 



j 
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In addition Pathfinders prepared by the Model Library staff, this list in- 
cludes compilation* - . : uted by the Tufts University Library staff and edited titles 

originally compiled as assignments by students enrolled in the Literature of Science and 
Technology course at Simmons College School of Library Science. 

The Pathfinders on "Black American Novels-ZOth Century" and "New England 
Transcendentalism" were compiled by the Model Library staff to demonstrate the adapt- 
ability of the Pathfinder concept to fields other than science and technology. These two 
Pathfinders have been used as samples for the social science and humanities literature 
specialists whom we have contacted with the proposal that they prepare Pathfinders in 
their subject areas of interest. 

Procedural delays and difficulties have interfered with full implementation of 
computer processing of Pathfinder text mentioned in the last semiannual report. After 
some successful tests of the input of Pathfinders to the computer and the successful out- 
put of one Pathfinder, we were unable to proceed with a larger sample because of changes 
in the input-output configuration of the peripheral equipment we were using. Specifically, 
the conversion to and from punched paper tape and magnetic tape became a bottleneck. 
Some temporary solutions emerged and were discarded because they were unreliable for 
day-to-day operations. New efforts to make this type of operation practical are in pro- 
gress because greater flexibility of format is possible, correction and updating will be 
less costly than manual methods, the results are more readable and therefore more use- 
ful than typescript. Photographic reductions of the Pathfinder prepared by computer in 
two fonts are shown in Figure III-6. We remain optimistic about the possibility of com- 
puter generated Pathfinder masters for future titles. Project T.I.P. (Technical Infor- 
mation Project) personnel have performed the tasks related to computer production of 
Pathfinders; their assistance is gratefully acknowledged. 

Librarians in cooperating libraries have questioned Project Intrex about the 
cost of producing Pathfinders adapted from the master copy furnished to them for users 
in their own libraries. Since the master is on "camera-ready" stock, the costs involved 
are those for finding and adding call numbers to the master, for preparing a printing 
plate, and printing it on stock drilled for a standard notebook. These costs may vary 
from one location to another but the figures we offer here are adequate for estimates 
and planning. 

A. Catalog search for local call numbers 
and recording of these call numbers. 

This task takes less time as more 
Pathfinders are done if a handlist of 
recurring titles and call-numbers is 

kept. Library assistant - paraprofessional; 

Twenty minutes at $5. 00/hour. $ 1.65 

B. Typing of library name and call numbers 
on the two sheets of the master Fathfinder. 

Skilled typist: 

Twenty minutes at $3. 30/hour. $ 1.10 
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Fig. Ill —6 Pathfinder “Thermal Pollution” produced by computer output in 

sans serif font “ Uni vers" (left) and serif font “Journal Roman” (right). 
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C. Printing 100 copies from photo-offset 

plate on drilled, high quality, fifty-pound 

offset paper stock, 2 sides. $10.7 5 

Printing 100 copies from photodirect 
plastic or paper plate on drilled, 

economy, fifty-pound stock, 2 sides. $ 4.20 

Printing 100 copies from electrostatic 
plate on driPed, economy, fifty-pound stock, 

2 sides. $ 2.60 

Quantities less than 100 copies of one title will reduce printing costs very 
slightly or not at all. However, storage of copies not used promptly may be expensive 
and unwieldy. In this case a shorter print run and more frequent printing may be bene- 
ficial. Use of heavier paper stock including cover stock or ledger stock will increase 
printing costs but the durability of the Pathfinder is also improved. 

Assembling the total costs from the three components (search, typing, printing), 
it can be seen that the lowest cost for 100 copies is $5.35; the highest is $13. 50. Within 
this range the large variables are the plate method used and paper stock selected. The 
combination of photodirect process plate and a high quality paper stock plus fixed charges 
for search and typing would result m a total cost of about $8. 00 for 100 copies. 

Recovery of the printing increment of the production costs may be achieved by 
offering them to the user for a fee smaller than the cost to him of photocopying & pirated 
copy — perhaps ten cents. 

If printing is eschewed for any reason, the library can make Pathfinders avail- 
able to the user by photocopying the master on legal-size paper and charging the user for 
two Xerox-type prints. In the experimental use of Fathfinders in the Barker Engineering 
Library they have been furnished free to users to obtain the v/idest base for study of user 
reactions. 

Further development of the cooperative program of Pathfinder compilation has 
occurred during this reporting period. Our goals have been to increase the number of 
Pathfinders produced and broaden the scope of Pathfinders to include topics in the social 
sciences and humanities as well as science and engineering. We aim to achieve these 
goals while maintaining established standards of quality and utility. 

We have continued the cooperative Pathfinder program with Mr. James Matarazzo 
and the Simmons science -technology students which v/as described in the last semiannual 
report. During this reporting period twenty-two student compilations on topics in elec- 
trical engineering, materials science, and pollution were edited by the Model Library 
staff and subsequently printed. These titles, together with the 15 previously reported, 
account for 37 of the 127 Pathfinders currently available. It is projected that 56 addi- 
tional science-technology Pathfinders will be prepared at Simmons this semester. 

The Simmons instructors in the Literature of the Social Sciences have also 
been contacted and they have indicated their interest in joining the cooperative program. 
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Final commitments in terms of numbers of student compilations to be submitted for pub- 
lication are pending because the Pathfinder assignments will be optional. 

In an attempt to expand the cooperative program, the Model Library staff wrote 
to faculty members at 49 institutions offering a graduate program in library science. Of 
the 114 faculty contacted, 36 responded favorably with an interest in having their students 
participate in the cooperative program. At the time of writing we have received firm 
commitments from 12 of these instructors that student compilations will be prepared and 
submitted. We expect to receive commitments from many of the remaining 24 instructors 
by the time this report is printed and distributed. On the basis of our correspondence 
thus far it is projected that we will receive between 250 and 300 Pathfinders to consider 
for publication this semester. See Figure IH-7 for details. 

In addition to involving library school faculty and students in the expansion of 
the Pathfinder program, our efforts with professional librarians have met with limited 
success. We have accepted eleven Pathfinders for publication from five libraries, and 
we have been notified that seven additional compilations are in progress. See Figure 
HI-8 for details. 

We have had responses from 7 other libraries indicating that they will br work- 
ing on compiling Pathfinders during this semester. Included in this group are Lehigh 
University, University of Houston, and Bath University of Technology. We will continue 
to pursue this avenue of approach with the full realization that, at many university and 
special libraries, the lack of available staff time for special projects of this sort pre- 
cludes production of Pathfinders in large numbers. In our contacts with cooperating 
institutions we have presented our role in the cooperative program of Pathfinder com- 
pilations as being threefold: (1) to a. as a clearinghouse to supply and/or record po- 
tential topics selected by the cooperating institutions; (2) to edit and prepare submitted 
compilation for publication as oversize printed masters; and (3) to act as a central 
distribution point in supplying oversize printed masters without call numbers? L cooper- 
ating institutions. To facilitate the Model Library Project’s function as a clearinghouse 
for Pathfinder topics, the staff has prepared a reserve file of topics available for com- 
pilation in mathematics, the physical sciences, engineering, the social sciences, fine 
arts, and the humanities. The staff, all of whom have different subject backgrounds, 
prepared lists of potential topics in their respective fields. The criteria were that the 
topics be of current research interest wi+hin their disciplines, that they have the degree 
of specificity required for meaningful Pathfinder compilation, and that they be repre- 
sented by bodies of literature. 

We have, though, encouraged cooperating institutions — both libraries and 
library schools — to select their own topics and tailor them for use in their own li- 
braries. We have suggested that, for maximum return on the investment of compilation 
time, it would be best for such locally selected topics to reflect (1) curriculum- and 
research-related areas of interest (which implies an identifiable body of Pathfinder users), 

- 66 - 



69 



Library Scnool 


Number of Pathfinder Topics 
Selected for Compilation 
(X - Exact number unknown) 


General Subject 
Disciplines Represented 


Simmons College 


56 


Science; Engineering 




X 


Social Sciences 




X 


Social Sciences 


Case Western Reserve University 


50 


Social Sciences 




26 


Medicine 




2 


Information Science 


Drexel University 


X 


Science; Engineering 


Emory University 


X 


English Literature 


Florida State University 


35 


Social Sciences 


Kent State University 


X 


Humanities 


McGill University 


9 


Science 


Rosary College (Illinois) 


40 


Humanities 


University of British Columbia 


13 


Science 


University of Denver 


33 


Social Sciences 


University of Illinois 


18 


Science; Engineering 



Fig. 111-7 Library schools compiling Pathfinders in the cooperative program. 



Library 


Number of Pathfinders: 


General Subject 
Disciplines Represented 


Accepted 


In Progress 


Duke University 

School of Engineering Library 


3 


2 


Engineering 


Tufts University Library 


4 




Art; Social Sciences 


Johns Hopkins University 
Applied Physics Laboratory 
Library 


3 




Physics 


Yale University 
Engineering and Applied 
Science Library 




2 


Physics 


Commonwealth of Massachusetts 
Department of Education 
Educational Reference Center 


1 


3 


Education 



Fig. 111-8 Libraries compiling Pathfinders in the cooperative program. 
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(2) the subject concentrations and strengths of library collections and staff, and (3) for 
library schools, the subject areas that best represent course emphasis and students' 
backgrounds and interests. 

However, when a cooperating institution requests that the Model Library staff 
supply topics for Pathfinder compilation, we ask to be appraised of the subject areas of 
interest. With this information we are better prepared to select from our reserve file 
topics th~t will be relevant to local compilers' interests. 

In addition to the reserve file of topics available for compilation, the Model 
Library staff also maintains records of topics for which compilations are either in pro- 
gress or have been completed. 

Before cooperating institutions begin, work on Pathfinders, we ask them to 
inform us of the topics that have actually been selected — either locally or from lists 
supplied by us. We then check the selections against our records of topics in progress 
and topics completed. We do this not to restrict or unduly influence compilers' choices 
but primarily to prevent duplicate compilations by any of the cooperating institutions. 

In our correspondence with cooperating institutions, we have indicated that a 
compiler may find it necessary to modify a topic whether it be one selected locally or one 
supplied by the Model Library staff. This procedure usually takes the form of narrowing 
or delimiting the topic, and we stress that the compiler should feel free to exercise a 
professional judgment. Our experience as compilers has demonstrated that the need for 
such a step becomes obvious only when one actually begins searching the literature. 

We have distributed to all cooperating institutions a summary of the guidelines 
and procedures which the Model Library staff has followed in compiling Pathfinders. 
Because the guidelines were developed with engineering Pathfinders foremost in mind, we 
have emphasized that the format described should be adapted for compilations in the social 
sciences and humanities- Our judgment is that adaptation consists primarily of tailoring 
the sections used in a Pathfinder to reflect the forms in which the literature of a discipline 
appears. We have indicated that compilers working in the social sciences and humanities 
should feel free to adapt the Pathfinder format according to the dictates of their disciplines. 

We have recently begun to print Pathfinders with a credit statement on the 
bottom of the verso. This credit identifies both the compiler and the institution at which 
the Pathfinder was prepared. 

Pathfinder revision continues to be a concern. We have added to our file of 
potential revisions not only suggestions obtained through feedback from Pathfinder users 
but also information which the Model Library staff becomes aware of in the course of 
compilation work and as new materials are received in the Barker Engineering Library. 

In addition, we have initiated a Pathfinder review program with the National Referral 
Center in the Science and Technology Reference Division of the Library of Congress. 
Finished Pathfinders are being submitted, on a continuing basis, to both referral spe- 
cialists and science reference librarians for review. Their comments concerning infor- 



mation sources and suggestions of additional citations are being incorporated into the 
revision file and will be utilized when our revision plans are made final. 

The methods of publicizing Pathfinders have continued along the lines previously- 
reported which include posters in departmental office and laboratory areas, notices printed 
in the Engineeing Library Bulletin , Pathfinder card entries under the pertinent subject 
headings in the Barker Engineering Library Subject Catalog, and notebooks of samples of 
available Pathfinders located at the library*s public service desks. In addition, Pathfind- 
ers have been incorporated into orientation programs and subject seminars conducted by 
the Barker Engineering Library staff. Some professors, too, have distributed explana- 
tory Pathfinder material to their classes and have recommended that their students use 
specific titles as bibliographic aids when compiling course -required papers. 

Response to the Pathfinder idea from librarians and others who have encoun- 
tered it outside M.I.T. has been enthusiastic and encouraging. Some samples of recently 
received unsolicited comments follow: 

National Referral Center - Library of Congress 
John A. Feulner, Senior Referral Specialist 

M They are, indeed, a marvelous idea. One wonders why 
nobody has thought of such a tool and put it into practice decades 
ago. I am most impressed by the whole concept. . . . Truly this 
new tool should prove most useful: telling people where to look 
and at the same time giving them something concrete. Actually, 
anybody, who can read, should be able to use and profit from the 
pathfinde rs. M 

Bath University Maurice Line, Director 

"The more I see your Pathfinders, the more I like them. 

They seem to me to be right on the ball, and have the advantage 
that every librarian can build up his own package to suit his own 
requirements. ... I am sold on the idea, and I shall want my own 
staff to start producing such things as soon as they possibly can." 

California Institute of Technology Ha raid Ostvold, Librarian 

"You have ingeniously reduced the guides to a level of 
simplicity that should encourage use by students, if that can be 
done at ail. " 

Colorado State University 

Marjorie Rhoades, Engineering Science Librarian 

"Your Pathfinders are indeed a welcome innovation. The 
approach you have taken to the perennial problem should be very 
fruitful. ... I certainly plan to adapt this technique for local use." 

Air University Library Robert Lane, Chief of Reader Services 

"Needless to say, I find them very intelligently done* and 
feel the format is exceedingly well worked out." 





International Business Machines, Inc . 

Eugene Jackson, Director, Information Retrieval and Library Services 

"I have been impressed by the potential of your Pathfinders. . . . 

In addition to the expected use in an industrial firm for briefing new 
employees or staff entering a new field of specialization, I see these 
as being crucially useful in developing countries that have to 
"make do" with basic collections including material in languages 
other than their native tongue." 

The primary distribution point for Pathfinders continues to be the Barker 
Engineering Library circulation desk. During the period August, 197 0 through January, 
1971, 480 Pathfinders were distributed on request to library users for personal use and 
retention. As previously reported, each of these Pathfinders had attached to it a self- 
addressed reply card on which the user was asked to record comments. 

Twenty-five cards have been returned with 28 comments. These comments 



can be categorized as follows: 

Uncritically favorable J6 

Favorable with reservations 4 

Suggested additions of material 5 

Unfavorable 1 

Additional topics suggested for Z 

compilation 



Of the four comments that were "favorable with reservations, " one suggested revising 
the scope note, one suggested that we list by name, prominent researchers in the topic 
field, one advised increasing Pathfinder publicity, and one thought that Pathfinder topics 
were too narrow. The one unfavorable comment indicated that the user had not found 
information on "computer utilities" in the Pathfinder on Time-Sharing. 

We had anticipated a low rate of return of the Pathfinder comment cards. 
Therefore, in addition to supplying the comment cards, we have used — and will 
continue to use during the next reporting period — another mechanism to obtain feed- 
back from users. 

Individuals who take Pathfinders are requested (but not required) to sign their 
name And address on a list maintained at the circulation desk. This list is subsequently 
used as a source for mailing a follow-up questionnaire (Fig. Ill- 9) to determine user 
reaction to Pathfinders. During the period September, 1970 through January, 1971, 105 
questionnaries were sent and 71 (68%) were returned. 
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1. How did you learn about Pathfinders? 

Engineering Library Subject Catalog 

__ Referred by Professor 
___ Engineering Library Bulletin 
___ Posters 

__ Sample notebooks in Engineering Library 

Other (specify) 

2. Why did you take them? 

__ Thesis research 
____ Course paper research 
___ Personal interest 

Other (specify) 

3A. Have you used them? 

__ Yes No 

B. If yes, which sections did you use? 

___ Subject catalog headings 
_ Frequently mentioned texts 
__ Handbooks, encyclopedias, and dictionaries 
__ Bibliographies 

Journal abstracts and indexes 
Journal titles 

State-of-the-art reviews and conference proceedings 

_ Technical report indexes 

4„ How helpful have the Pathfinders been? 

Very helpful 
Fairly helpful 
Not helpful 

5. Can you suggest ways to improve Pathfinders? 



6. Can you suggest additional topics? 



Name: 

Extension: 



Fig. 1 1 1-9 Questionnaire sent to Individual Pathfinder users. 





; responses to question #1 were as follows: 
#1. How did you learn about Pathfinders? 


__ # 

Responses 




No. 


% 


Engineering Library subject catalog 


27 


(38%) 


Referred by Professor 


14 


(20%) 


Engineering Library Bulletin 


9 


(13%) 


Posters 


5 


( 7%) 


Sample notebooks in Engineering Library 


13 


(18%) 


Librarian 


7 


(10%) 


Other 


4 


( 6%) 


(*N = 71; eight respondents checked two 


answers 


) 



The largest group of respondents (38%) were introduced to Pathfinders through 
the color-coded cards entered in the Barker Engineering Library Subject Catalog under 
relevant headings. However, a significant combined percentage of respondents (40%) 
learned about Pathfinders from professors whom the Model Library staff had contacted 
directly, from announcements in the biweekly Engineering Library Bulletin , and from 
posters selectively located in departmental areas. This suggests to us that the subject 
catalog consititutes as good a Pathfinder mechanism as any for reaching presumably re- 
gular library users; yet, the promotion of Pathfinders by means external to the library is 
a mechanism which has as much potential for reaching another identifiable group within 
the M.I.T. community. Conceivably, this latter group includes persons who do not have 
preestablished library habits and are, therefore, most in need of assistance when they 
have subject literature requirements. 

The responses to question #2 indicated the reasons Pathfinders were taken to 
be as follows: 

#2. Why did you take them? Responses 





No. 


_% 


Thesis research 


14 


(20%) 


Course paper research 


34 


(48%) 


Personal interest 


21 


(30%) 


Project research 


8 


(11%) 


Non-MIT company research 


2 


( 3%) 


Other 


3 


( 4%) 



(*N = 71; eleven respondents checked two answers) 

The percent of responses (30%) indicating that Pathfinders were taken for 
personal (i. e. , non-research, non- curriculum) interests was higher than expected. 

We do not know how much the curiosity factor influenced these responses. 

The percent of responses (20%) indicating an interest in Pathfinders in connec- 
tion with thesis research was lower than expected. This may be a reflection of a common 
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"searching" practice among thesis writers of consulting, almost exclusively, specific 
materials recommended by departmental advisors and/or subject collections scattered 
throughout departmental offices. 

The high percent of Pathfinder interest (48%) relative to course paper research 
reinforces our opinion concerning the value of having instructors distribute to students 
introductory material indicating the availability of specific Pathfinders relevant to course 
content. 

The responses to questions #3A and #3B concerning the Fathfinder sections 
actually used we re as follows: 

#3A. Have you used them Responses* 





No. 


% 


Yes 


60 


(85%) 


No 


10 


(14%) 


Not yet 


1 


( 1%) 



(*N =71) 



#3B, If yes, which sections did you use Responses* 





No. 


Jb 


Subject catalog headings 


16 


(27%) 


Frequently mentioned texts 


36 


(60%) 


Handbooks, encyclopedias, dictionaries 


13 


(22%) 


Bibliog raphie s 


13 


(22%) 


Journal abstracts and indexes 


24 


(40%) 


Journal titles 


17 


(28%) 


State-of-the-art reviews and 
conference proceedings 


18 


(30%) 


Technical report indexes 


17 


(28%) 



(*N = 60) 

The responses to these questions indicate that the majority of Pathfinders 
taken (85%) were actually used — the curiosity factor notwithstanding. In addition, the 
indications are that all sections constituting the Pathfinder format are used with sufficient 
frequency to warrant continued inclusion. Concerning the Pathfinder sections, there has 
not been any negative user reaction to the sequence in which they are arranged. 

The responses to question #4 concerning the Pathfinders 1 utility were as follows: 

#4. How he lpful have the Pathfinders been Responses* 





No. 




Very helpful 


26 


(43%) 


Fairly helpful 


28 


(47%) 


Not helpful 


6 


(10%) 



(*N = 60) 
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USER'S REASON FOR TAKING PATHFINDER 


Thesis 

Research 


Course Paper 
Research 


Personal 

Interests 


Other 


Project 

Research 


No. % 


No. % 


No, % 


No. % 


No. % 


Very helpful 


3 (23%) 


18 (60%) 


6 (33%) 


1 (50%) 


3 (50%) 


Fairly helpful 


8 (62%) 


9 (30%) 


12 (67%) 


1 (50%) 


2 (33%) 


Not helpful 


2 (15%) 


3 (10%) 


0 


0 


1 (17%) 


Total 


13 

1 


£ 


18 


2 


6 



Fig. 111-10 Correlation between user's reason for taking Pathfinder 
and user's judgment of utility. (9 respondents took 
Pathfinders for two reasons) 





NUMBER OF PATHFINDER SECTIONS USED 


8 


7 


6 


5 


4 


3 


2 


1 


No. % 


No. % 


No. % 


No. % 


No. % 


No. % 


No. % 


No. % 


Very helpful 


3 (100%) 


0 


0 


1 (50%) 


3 (36%) 


2 (25%) 


11(58%) 


6 (32%) 


Fairly helpful 


0 


0 


1(100%) 


1 (50%) 


5 (64%) 


6 (75%) 


7(37%) 


8 (42%) 


Not helpful 


0 


0 


0 


0 


0 


0 


1 (5%) 


5 (26%) 


Total 


3 


0 


1 


2 


8 


8 


19 


19 



Fig. 111-11 Correlation between number of sections used 
and user's judgment of utility. 
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These responses indicate to us — in a general way, at least — that, for the 
majority of respondents (90%), the Pathfinders are fulfilling their stated objective".. . to 
help library users begin to fii.d information...." Upon analysis, the 10% "not helpful" 
responses are really not disturbing* One negative respondent indicated that he had, in 
fact, not used the Pathfinders much but he did". . . see high potential for them." For the 
most partj, the remainder of the negative responses represent the reactions of users who 
selected Pathfinders not pertinent 10 their subjects or who expected Pathfinders to be more 
of a bibliography* 

With the assistance of Mrs* Susan F. Brown, Project Intrex psychologist, a 
completely new and refined questionnaire for obtaining user feedback is currently being 
prepared. It is expected that this question concerning users' subjective measures of 
Pathfinders' utility will be broadened to include more gradations of opinion. 

On the basis of questionnaire responses, correlations have been obtained con- 
cerning users' judgments of Pathfinder utility in terms of both the reasons for which Path- 
finders were taken (Fig. Ill- 1 0) and the number of Pathfinder sections used (Fig. Ill- 1 1 ) . 

The comments made by respondents on improving Pathfinders did not indicate 
to us any major areas of weakness. The suggestions can be categorized as follows: 

#5. Can you suggest ways to improve Pathfinders? Responses* 

No. % 



No suggestions 


♦10 


(56%) 


Addition of type of material 


9 


(13%) 


Improve layout 


4 


(6%) 


Publicize 


4 


(6%) 


Update Pathfinders 


3 


(4%) 


Finer subject breakdown of material 


3 


(4%) 


Specific additions suggested 


1 


(1%) 


Identify outstanding researchers 


1 


(1%) 


Irrelevant suggestions 


8 


(11%) 



(*N = 71; two respondents made two suggestions) 

During the next reporting period the main efforts of the Model Library staff 
regarding Pathfinders will be concentrated on: 

(1) increasing the number of available Pathfinders through 
the development of the cooperative compilation program,* 

(2) increasing the distribution of printed masters to 
cooperating institutions for use in their own libraries; and 

(3) increasing user feedback concerning Pathfinders by means 
of the refined questionnaire now in preparation. 
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D. 



USER PREFERENCE STUD Y 



Mr. C. H. Stevens 
Mr. J.J* Gardner 
Mr. J. M. Kyed 



With a substantial period of operation carefully recorded in the Barker Engi- 
neering Library Microfilm Service Area there is no evidence for changing the widely 
held view that a majority of library users have a strong preference for hard copy over 
film copy whenever they are equally available. But the "majority" may not be quite as 
large or quite as insistent as it has been in the past. As the collection of fiche has be- 
come more comprehensive and the fiche readers more available, there has been a small 
but significant increase in the use of these materials. Since the number of requests for 
copies of fiche and for hard copies from fiche has been small, the concentration of effort 
has been on building the collection and on improvement of the access options available to 
the user. 

The collection now has nearly 30, 000 fiches. During the last six months the 
additions have included a collection of manufacturers' catalogs representing over 400 
companies, significantly long runs of thirty-four journal titles, several thousand Mil. T. 
theses in engineering, and all of the materials indexed for the Intrex augmented catalog 
experiments. Take i together, the new acquisitions double the collection held at the time 
of the last semiannual progress report. It is reasonable to expect that this larger collec- 
tion will have wider use since the patron will ha\ j greater expectancy of obtaining the 
desired item. The emphasis on collection will continue during the next six months. 

During this period the thesis filming project will be concluded for all titles from 1946 - 1969. 
A large percentage of M.I-T. 197 0 engineering theses will also be complete by September. 
Purchases of available journals, society papers, conference reports, and monographs on 
fiche will continue and a program of filming other heavily used current journal issues will 
be gin. 

To improve user access to the fiche two new types of microfiche readers have 
been added to those already available and reported earlier. One of the readers, designed 
for library, office, dormitory, or classroom use, by the Electronic Systems Laboratory 
at M.I. T. under a contract from the United States Office of Education features a motor- 
driven fiche carrier with remote control operation and projection of the film image on a 
wall screen. Installation of this unit in the Barker Engineering Library occurred in 
January 1971 and the users who have tried it are pleased with both the image quality and 
the simplicity of operation. Since the image is large and bright there has been no com- 
plaint about eye fatigue by our users. The second new reader is the widely-advertised, 
eight-pound portable microfiche viewer made by DASA in Andover, Massachusetts. This 
reader, also developed under Office of Education sponsorship, has just become available 
to us in quantity and first experiments in loaning units to readers will occur about the 
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time this report is distributed. The opportunity exists with the receipt of the portable 
reade rs to test the view that use of fiche increases in proportion to the easy availability 
of readers for fiche. 

A program of displays and demonstrations calling attention of casual library 
users to the advantages of microfiche began during the opening of classes in late September. 
Increased awareness to the growing collection is, we believe, a key to increased utili- 
zation and therefore the effort to promote use by alerting patrons to the collections and 
how to use them will continue. Two of the displays used for microfiche publicity are 
reproduced in Figure III-1Z. 




Fig. 111-12 Bulletin board displays used to draw attention to microfiche services. 
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IV. PROJECT INTREX STAFF 

A. PROJECT OFFICE 
Professor Carl F.J. Overhage, Director 
Mr. Charles H. Stevens 



B. ELECTRONIC SYSTEMS LABORATORY 



Professor J, Francis Reintjes 
Mr. Alan R. Benenfeld 
Mr. Larry E. Bergmann 
Mr. Joseph Bosco 
Mrs. Susan Brown 
Professor Lynwood S. Bryant 
Mr. Dennis W. Buckley 
Mr. Peter Campoli 
Mr. Yiu T. Chan 
Miss Margaret A. Flaherty 
Mr. Robert Goldschmidt 
Mr. Daniel J.. Griffin 
Mr. Charles E. Hurlburt 
Miss Margaret A. Jackson 
Mr. James E. Kehr 



Mr. Donald R. Knuds on 
Mr. Peter Kugel 
Mr. Stephen G. Kuyamjian 
Miss Linda A. Langille 
Miss Lucy T. Lee 
Mr. Richard S. Marcus 
Mr. Patrick L. Martin 
Miss Mary M. McMillin 
Miss Virginia A. Miethe 
Mr. Michael K. Molnar 
Professor James K. Roberge 
Mr. JohnO. Silvey 
Miss Frances Spigai 
Mr. Jean R. Ward 



C . BA RKER ENGINEERING LIB RA R Y 



Miss Rebecca L. Taggart, Head 
Mrs, Marjorie Chryssostomidis 
Miss Barbara C. Darling 
Mrs. Kate Herzog 
Miss Carol L. Keator 



Mr. James M. Kyed 
Miss Helen Magedson 
Miss Susan Nutter 
Miss Mary Pensyl 
Mr. David C. Van Hoy 



D. MODEL LIBRARY PROGRAM 



Mr. Jeffrey J. Gardner 
Miss Marie P. Canfield 



Mrs . Elizabeth King 
Miss Katherine C. Todd 



V. 



CURRENT PUBLICATIONS 



A. REPORTS 
None 

B. BOOK CHAPTERS, JOURNAL ARTICLES AND CONFERENCE PAPERS 

Marcus, R. S. , Benenfeld, A. R. , and Kugel, P. , "The User Interface for the Int rex 
Retrieval System. " Presented at the Workshop on the User Interface for Interactive 
Search of Bibliographic Data Bases, Palo Alto, California, January 14-15, 1971. 
Proceedings to be published by A FIPS Press. 

Lovins, J. B. , "Error Evaluation for Stemming Algorithms as Clustering Algorithms. " 
Journal of the American Society for Information Science, Yol. 22, No. 1, January, 

1971, pp 28-40. 

Stevens, C. H. , "Specialized Microform Applications in an Academic Library. M 
Presented at the University of Denver, Denver, Colorado, December 7, 1970. 

(To be published in the Proceedings. ) 

Overhage, Carl F. J. , "Directions for the Future. " Presented at Collaborative 
Library Systems Development Conference, New York, N. Y. , November 10, 1970. 

(To be published in Conference Proceedings.) 

C. THESES 
None 

D. MISCELLANEOUS PRESENTATIONS 

Stevens, C. H. , "Project Int rex and Engineering Library Services. " Presented 
at Boston University, Boston, Massachusetts, January 12, 1971. 

Stevens, C. H. , "Project Int rex at Midstream. " Presented at the University of 
Illinois, Urbana, Illinois, November 20, 1970. 

Stevens, C. H. , "The Sky is Not the Limit." Presented at Honeywell Corporation 
Executive Seminar, Concord, Massachusetts, November 16, 1970. 

Stevens., C. H. , "Science and Technology Information Services in the Academic 
Library . " Presented at North Carolina Central University, Durham, North 
Carolina, October 21, 1970. 

VI. PAST PUBLICATIONS — October, 1969 through 15 September 1970. 

A. REPORTS 

Goto, Nobuyuki, "A Translator Program for Displaying a Computer Stored Set 
of Special Characters. " ESLrR-429, July, 1970. 

Kusik, R. L. , "A File Organization for the Intrex Information Retrieval System 
on the 360/67 CP/CMS Time-Sharing System. " ESL-TM-415, January, 1970. 

Lovins, J. B. , "Error Evaluation for Stemming Algorithms as Clustering Algorithms. " 
ESL-R-411, December, 1969. 
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Haring, D. R. , M The Augmented -Catalog Console for Project Intrex (Part II). " 
ESL-TM-410, December, 1969. 

Project Intrex Staff, Semiannual Activity Report, 15 September 1970. 

Project Intrex Staff, Semiannual Activity Report, 15 March 1970. 

B. BOOK CHAPTERS, JOURNAL ARTICLES AND CONFERENCE PAPERS 

Reintjes, J. F. , "Recent Experiments with the Project Intrex Information Storage 
and Retrieval System. " Gordon Conferences, New London, New Hampshire, 

July 16, 1970. 

Knudson, D. R. and Vezza, A., "Remote Computer Display Terminals . " Conference 
on Computer Handling of Graphical Information sponsored by SPSE, NMA, and SID, 
Newton, Mass., July 9-10, 1970, Proceedings , pp 249-268. 

Stevens, C. H. , "New Whine in Olde Bottles." Presented at American Library Asso- 
ciation National Convention, Detroit, Michigan, July 2, 1970. 

Stevens, C. H. , "Point-of-use- Instruction in Libraries. " Presented at American 
library Association National Convention, Detroit, Michigan, June 29, 1970. 

Stevens, C. H. , "Destination Shangri- La, First Stop Erewhon. " Presented at 
American Society for Engineering Education National Conference, Columbus, Ohio, 

June 25, 1970. 

Roberge, J. K. and King, P. A., Jr., "An Economical Approach to High-Speed Charac- 
ter Generation and Display. " 1970 Society for Information Display Symposium, New 
York, N. Y., May 26-28, 1970, Digest of Fapers , pp 104-105. 

Stevens, C. H. , "Experiments with Microfiche in an Academic Library." Presented 
at the National Microfilm Conference, San Francisco, California, April 27, 1970. 

Reintjex, J. F. , "Hardware, " as related to "Issues and Problems in Designing a 
National Program of Library Automation. " Library Trends, Vol. 18. No. 4. April. 
1970, pp 503-519. 

Overhage, C. F. J. and Reintjes, J. F. , "Computers in Libraries, Servant or Savant " 
Presented at American Society for Information Science, New England Chapter Meeting, 
March 25, 1970. 

Knudson, D. R. , "Image Storage and Transmission for Project Intrex, " Conference 
on Image Storage and Transmission for Libraries, National Bureau of Standards, 
Gaithersburg, Maryland, December 1-2, 1969. 

Overhage, C. F. J. , "Information Networks, " Chapter 11 in Annual Review of Int’or- 
>ggtion Science and Technology, Vol. 4, Carlos A. Cuadra, Editor. E n eye lo pe dia 
Britannica, Inc., Chicago, 1969. 



C. THESES 

Goto, Nobuyuki, "A Translator Program for Displaying a Computer Stored Set of 
Special Characters. " M. S. thesis, Electrical Engineering Department, Massachusetts 
Institute of Technology, July, 1970. (Also Electronic Systems Laboratory Report 
ESL-R-429.) 

Kusik, R. L , "A File Organization for the Intrex Information Retrieval System on the 
360/67 CP/CMS Time-Sharing System." M.S. Thesis, Electrical Engineering Depart- 
ment, M.I. T. , November, 1969. (Also Electronic Systems Laboratory Technical 
Memorandum ESL-TM-415 
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VII. 



PAST PUBLICATIONS — 1966 through September, 1969 



A. REPORTS 



Jagodnik, A. J., Jr., "Performance Evaluation of Image Storage and Transmission 
Systems. " ESL-R-391, June, 1969. 

King, P. A., Jr., "A Novel Solid State Character Generator. " ESL-TM-386, June, 
1969. 



Lufkin, R. C., "Determination and Analysis of Some Parameters Affecting the Subject 
Indexing Process. " ESL-R-364, September, 1968. 

Kampe, W. R. , "Pre-Indexing by Machine. " ESL-R-355, July, 1968. 

Lovins, J. B., "Development of a Stemming Algorithm. " ESL-TM-353, June, 1968. 



Benenfeld, A. R., "Generation and Encoding of the Project Intrex Augmented Catalog 
Data Base." ESL-R-360, August, 1968. This report is based upon a paper presented 
at the 6th Annual Clinic on Library Applications of Data Processing, University of 
Illinois, Urbana, Illinois, May 7, 1968. 

Haring, D. R. , and Roberge, J. K. , "The Augmented Catalog Console for Project 
Intrex. " ESL-TM-323, October, 1967. 

Gronemann, U. F. , Knudson, D. R., a«id Teicher, S. N., "Remote Text Access for 
Project Intrex. " ESL-TM-312, July, 1967. This report is based upon a conference 
paper presented at the National Microfilm Association Convention, Miami Beach, 
Florida, April 26-28, 1967. 



Benenfeld, A. R., Gurley, E. J. , and Rust, J. E., "Cataloging Manual. " ESL-TM-3 03, 
February, 1967. 

Project Intrex Staff, Semiannual Activity Report, 15 September, 1969. 

Project Intrex Staff, Semiannual Activity Report, 1 5 March, 1969. 

Project Intrex Staff, Semiannual Activity Report, 15 September, 1968. 

Project Intrex Staff, Semiannual Activity Report, 15 March, 1968. 

Project Intrex Staff, Semiannual Activity Report, 15 September, 1967. 

Project Intrex Staff, Semiannual Activity Report, 1 5 March, 1967. 

Project Intrex Staff, Semiannual Activity Report, 15 September, 1966. 

Project Intrex Staff, Semiannual Activity Report, 15 March, 1966. 

B. JOURNAL ARTICLES AND CONFERENCE PAPERS 



Stevens, C, H. , "Will the Library Sink or Swim — A Buoyant Response. " Presented at 
the American Chemical Society National Meeting, New York, New York, September 8, 
1969. 

Reintjes, J. F. , "The Use of Multi-access Computers for the Management and Control 
of Professional Literature, " Fourth Congress of the International Federation of Auto- 
matic Control, Warsaw, Poland, June 16-21, 1969. 
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Knudson, D. R., and Teicher, S. N. , "Remote Text Access in a Computerized Library- 
Information Retrieval System, " 1969 Spring Joint Computer Conference, Boston, Mass., 
May 14-16, 1969, AFIPS Conference Proceedings , Vol.34, pp, 475-481. 

Marcus, R. S. , Kugel, P. , and Kusik, R. L., "An Experimental Computer- Stored, 
Augmented Catalog of Professional Literature, " 1969 Spring Joint Computer Conference, 
Boston, Mass., May 14-16, 1969, AFIPS Conference Proceedings , Vol. 34, pp. 461-472. 

Reintjes, J. F., "System Characteristics of Intrex, " 1969 Spring Joint Computer Con- 
ference, Boston, Mass., May 14-16, 1969, AFIPS Conference Proceedings , Vol. 34, 
pp. 457-459. 

Stevens, C. H. , "Book Publishing, A Look to the Future." Presented at the Association 
of American University Presses Conference on Information Science and Publishing, 
University of Chicago, April 22, 1969. 

Benenfeld, A. R. , "Generation and Encoding of the Project Intrex Augmented Catalog 
Data Base. " Proceedings of the May 6, 1968 Clinic on Library Applications of Data 
P roce ssing , Urbana, Illinois, University of Illinois, Graduate School of Library Science, 
1969, pp. 155-198. (Also issued as Electronic Systems Laboratory Report ESL-R-360, 
August, 1968.) 

Haring, D. R., "Computer-Driven Display Facilities for an Experimental Computer- 
Based Library. " 1968 Fall Joint Computer Conference, San Francisco, California, 
December 9-11, 1968. AFIPS Conference Proceedings , Vol. 33, pp. 255-265. 

Haring, D. R., and Roberge, J. K. , "Display Techniques for an Experimental 
Computer-Based Library. " UAIDE Conference, San Francisco, California, October 
28-31, 1968. 

Knudson, D. R., Teicher, S. N. , Reintjes, J. F., Gronemann, U. F. , "Experimental 
Evaluation of the Resolution Capabilities of Image-Transmission Systems. " Informa - 
tion Display - September/October, 1968. 

Stevens, C. H. , "Then Beggars Would Ride. " Engineering Joint Management Council 
Conference, Philadelphia, Pennsylvania, September 30, 1968. 

Haring, D. R., "A Display Console for an Experimental Computer-Based Augmented 
Library Catalog. " 1968 ACM National Conference and Exposition, Las Vegas, Nevada, 
August 27-29, 1968. (To Appear in Conference Proceedings.) 

Reintjes, J. F., "Project Intrex, (A Re search Project under Development). " Presented 
at the Conference on Computers and the University , Technical University of Berlin, 

July 22-August 2, 1968. 

Benenfeld, A. R., "Data Encoding for the Project Intrex Augmented-Catalog Experi- 
ments." Presented at the American Documentation Institute Convention, User Discus- 
sion Group on Emerging Machine -Readable Tape Services, New York, October 25, 1967. 

Haring, D. R., "A Terminal for an On-Line Interactive Retrieval System. " Presented 
at the IEEE Workshop on Advanced Computer Peripherals, Lake Arrowhead, California, 
August 25-27, 1967. 

Overhage, Carl F. J., "Science Libraries: Prospects and Problems. 11 Science , 155 , 

17 February, 1967, pp. 302-806. 

Overhage, Carl F. J., "Plans for Project Intrex. " ^Science , 1952 , 20 May, 1966, pp. 
1032-1937. 
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C., THESES 



King, P. A., Jr,, "A Novel Solid State Charafcter Generator, " M. S. Thesis, Electrical 
Engineering Department, Massachusetts Institute of Technology, June, 1969* Also 
Electronic Systems Laboratory Report ESL-TM-386, June, 1969. 

Jagodnik, A. J., "Performance Evaluation of Image Storage and Transmission Systems." 
M.S. Thesis, Massachusetts Institute of Technology, February, 1969. 

McKenzie, P. F., "A Flying-Spot- Scanner Character Generator. " M.S. Thesis, 
Massachusetts Institute of Technology, February, 1969. 

Kampe, W. R. , "Pre -Indexing by Machine. " M.S. Thesis, Electrical Engineering 
Department, Massachusetts Institute of Technology, June, 1968. Also Electronic 
Systems Laboratory Report ESL-R-355, published July, 1968. 

Lufkin, R. C., "Determination and Analysis of Some Parameters Affecting the Subject 
Indexing Process. " B.S. Thesis, Massachusetts Institute of Technology, June, 1968. 
Also Electronic Systems Laboratory Report ESL-R-364, published September, 1968. 

Domercq, R. J., "A Machine-Aided Thesaurus Generation System. " M.S. Thesis, 
Massachusetts Institute of Technology, September, 1967. 

Stuntz, S. C., "Inputting Process to an Augmented Library Catalog -- An Efficiency 
Study." B.S. Thesis, Massachusetts Institute of Technology, June, 1967. 



D. MISCELLANEOUS PRESENTATIONS 



Alan R. Benenfeld 

"The Project Intrex Augmented Catalog. " Syracuse University, Graduate School of 
Library Science, May 22, 1969. 

Gary L. Benton 

Panel on New Hardware Developments, ADI Annual Meeting, Santa Monica, California, 
October 5, 1966. 

"Project Intrex -Information Transfer Experiments at M.I.T. " Raytheon Institute of 
Bedford, Bedford, Mass., February 15, 1966. 

"Project Intrex: Plans and Progress. 11 Massachusetts Association of Medical Record 
Librarians, Boston, Mass., January 26, 1966. 

Carl F. J. Overhage 

"Information Transfer Experiments at M.I.T." International Federation for Information 
Processing, IFIF 68 Congress, Edinburgh, Scotland, August 6, 1968, 

"Information Transfer in the Library of the Future. " Panel of the International Com- 
munications Conference on "The Evolving Relationship Between the Publisher and the 
Electronic Firm- -So ft ware and Hardware, " IEEE, Philadelphia, Penn., June 12, 1968. 

"Plans for Project Intrex. " Symposium on Mechanized Information Services in the 
Library, UCLA, March 29-31, 1968. 

"Project Intrex- Plans and Progress. " Harvard University Librarians Seminar, 
Cambridge, Mass., May 17, 1967. 
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"Project Intrex and the Future of the- Book. 11 American Book Publishers' Association, 
Arden House, Harriman, New York, March 16, 1967. 

M.I.T. Center for Advanced Engineering Study, Cambridge, Mass., November 29, 1966. 

American Cyanamid Company, Librarians and Information Specialists Meeting, Princeton., 
New Jersey, October 20, 1966. 

M.I.T. Sloan School of Management, Cambridge, Mass., October 13, 1966. 

"Plans for Project Intrex at M.I.T. " 

Panel on National Information Issues and Trends, ADI Annual Meeting, Santa Monica, 
California, October 6, 1966. 

Panel on Information Systems Applications, ADI Annual Meeting, Santa Monica, 

California, October 5, 1966. 

Association of Research Libraries, New York, New York, July 9, 1966. 

AF1PS Spring Joint Computer Conference, Boston, Mass., April 28, 1966. 

Rochester Computer Systems Association, Rochester, New York, April 26, 1966. 

Optical Society of America and American Chemical Society, (Rochester sections) 
Rochester, New York, March 1, 1966. 

M.I*T. Lincoln Laboratory, Lexington, Mass., February 8, 1966. 

American Physical Society, New York, New York, January 29, 1966. 

American Library Association, Chicago, Illinois, January 26, 1966. 

American Association for the Advancement of Science, Berkeley, California, 

December 29, 1965. 

American Documentation Institute, Cambridge, Mass., October 28, 1965. 

"Plans for Information Transfer Experiments at M.I.T. " 

Charles H. Stevens 

"Project Intrex - A Progress Report. " MITRE Corporation Information Systems 
Seminar. Bedford, Mass., November 5, 1966*. 

"Console Development at Project Intrex. " Special Libraries Association, Los Angeles, 
California, June 5, 1968. 

"Books and Other Paraphernalia - A Forward Look. " National Endowment for the 
Humanities, Boston, Mass., May 1 0, 1968. 

"Engineering Libraries and Information Systems. " Center for Advanced Engineering 
Study, M.I.T., Cambridge, Mass., April 16, 1968. 

"Project Intrex - Plans and Progress. " New York Chapter, Special Libraries Associa- 
tion, New York, New York, February 6, 1968. 

"Project Intrex. " Gordon Research Conference on Scientific Information, New London, 
New Hampshire, July 19, 1967. 

"New Developments in Libraries. " State University of New York (Albany), April 4, 1967. 
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